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COMBINATORIAL  ANALYSIS  OF  LINEAR  PROGRAMS 
(An  Introduction  to  Underlying  Theory) 

by  A.  W.  Tucker 
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A  MINIATURE  EXAMPLE  OF  LINEAR  PROGRAMMING.   A  linear  program  is  a  problem  of 
"optimizing"  (i.e.,  maximizing  or  minimizing)  a  linear  "objective"  function  of 
many  (real)  variables  subject  to  a  system  of  linear  "constraints,"  each  of  which 
is  a  linear  inequality  or  linear  equation.  The  following  is  an  example,  in 
miniature  : 

Maximize  the  objective  function 

(0)  \   -  3m-  +  9 
subject  to  the  constraints 

(1)  -\  +  2\i   -  2  >  0 

(2)  \  -  n  +  3  >  0 

(3)  H  -  1>  0 
(k)  \  -  2n  +  5  >  0 
(5)    -\  +  |i  -  2  >  0  . 

This  miniature  linear  program  can  readily  be  analyzed  graphically.     In  a 
\,H-coordinate  plane  (see  Figure  1)  we  plot  the  "halfplanes"   (l)-(5).      I11  each 
case  the  halfplane  (which  includes  its  boundary  line)  is  labelled  by  its  number 
along  its  boundary  line  on  the  side  of  the  line  in  which  the  halfplane  lies:     for 
example,  the  label  (l)  appears  along  the  line     -\  +  2u  -  2  =  0     within  the  half- 
plane     -\  +  2\x  -  2  >  0.     The  set  of  feasible  points     (\,n)     satisfying  all  five 
constraints  is  a  quadrilateral  determined  by  (2),    (3),    (^),    (5).     The  first  con- 
straint is  inactive,   as  it  happens,  because  the  feasible  set  lies  entirely  within 
the  "open  halfplane"     -\  +  2[i  -  2  >  0.     The  objective  function,  which  is  to  be 
maximized,   is  represented  by  seme  particular  contour  (or  level)  lines  along  which 
it  takes  values  as  indicated.     We  see  that  the  desired  maximum  appears  to  occur  at 
the  point  of  intersection  of  lines  (3)  and  (5),  namely  at  the  point  35,  with 
coordinates     \=  -1,   n  =  1,     where  the  objective  function  takes  the  value     5. 
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FIGURE  1.  Graphical  analysis  of  the  miniature  linear  program, 


SYSTEMATIC  ANALYSIS  OF  EXAMPLE.     The  foregoing  graphical  method  of  solving 
a  linear  program  does  not  generalize  "beyond  two  or  three  variables,  whereas  linear 
programs  involving  scores  or  hundreds  of  variables  are  not  uncommon  in  actual 
practice.     So,   starting  afresh,  we  outline  a  more  formal  systematization  of  our 
example  which  does  generalize  to  large-scale  problems. 

To  this  end,  we  introduce  the  "schema" 


J  "j        "  O       •'9          ll          S 

\ 

-110       1-11 

u 

2-1       1-2       1-3 

1 

-2      3-1       5-2      9 

=0 
=0 
=*v 
=X;L  =x2  =x3  =x^  =x5  =u 

consisting  of  a  matrix  of  three  rows  and  six  columns  inside  the  box  and  certain 
"labels"     \f\xfl,  etc.,     around  the  four  margins  of  the  box.     The  six  signs  of 
equality  along  the  bottom  of  the  box  indicate  a  "column  system"  of  linear  equations 

-\  +  2u  -  2  =  y^ 
\  -     u  +  3  ■  x2 
u  -  1  =  x3 
\  -  2u  +  5  =  x^ 
-\  +     [x  -  2  -  x5 
X  -  3  m-  +  9  ■  u 
obtained  by  forming  inner  products  of    \,n,l    with  the  columns  of  the  matrix  and 
setting  these  inner  products  equal  to     x1,x2,Xo,Xr,x(.     and    u.     The  three  signs  of 
equality  at  the  right-hand  margin  of  the  box  indicate  a  "row  system"   of  linear 
equations 
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-  y1  +   v2  +  y^  -  y^  +  i  =  o 

2yx  -  y2  +  y3  -  2yu  +  y^  -  3  »  o 
-2y1  +  3y2  -  y3  +  5y^  -  2y^  +  9  =  v 
obtained  by  forming  inner  products  of  the  rows  of  the  matrix  with  y,,yp,y.,,yL,ycs,l 
and  setting  these  inner  products  equal  to  0,0  and  v. 
We  point  out  that 

xlyl  +  x2y2  +  x3y3  +  x4yU  +  x5y5  +  u 
becomes 

\(-  y1  +  y2  +  y^  -  y5  +  i) 
+n(  2yx  -  y2  +  y3  -  2y^  +  y^  -  3) 
+  (-Zy-L  +  3y2  -  y3  +  5y^  -  2y5  +  9) 

by  substituting  for     at,, .  ..,x,_,u     from  the  column  system.     However,  by  the  row 
system  this  reduces  simply  to     v.     Note  that  this  "key  equation" 

xlyl  +  x2y2  +  x3y3  +  Vk  +  x5y5  +  u  =  v 
(in  which    \t\i     do  not  appear  explicitly)  is  obtained  by  setting  the  inner  product 

of  the  bottom  marginal  labels  with  the  top  marginal  labels  equal  to  the  inner 

product  of  the  left-hand  marginal  labels  with  the  right-hand  marginal  labels. 

Our  problem     is  to  maximize     u     for  feasible     \     and     \i,     i.e.,   for    \     and 
|i    which  make  the  five     x's     nonnegative,  where  the     x's     and    u     denote  the  values 
of  the  linear  functions  specified  by  the  column  system  of  equations.     We  observe 
that 

x^  +  xgy2  +  x3y3  +  x^  +  x^  +  u  = 

(-\42u-2)yx  +  (\-u+3)y2  +  (n-i)y3  +  (\-2u+5)yu  +  (-\+u-2)y5  +  (\-3n+9)  >X-3n+9 

if    \,|J.     are  feasible  and  the  (Lagrange)  "multipliers"     y1,y2,y^,y^,ye     sore  non- 
negative.     If  the     y's     and    v     also  satisfy  the  row  system  of  equations  then 
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v  >  \  -  3|i  +  9  . 

That  is,  v  is  an  upper  bound  on  u  =  \-3n+9  for  feasible  \,n,  provided 
yvvP>yVyVy5  are  feasit,lej  !•••!  are  nonnegative  and  with  v  satisfy  the  row 
system  of  equations.  Thus,  if  we  can  find  a  feasible  pair  \,u  and  a  feasible 
quintuple  of  y's  such  that 

v  =  u  , 
our  problem  is  solved. 

The  fundamental  existence  theorem  of  linear  programming  (not  to  be  discussed 
here)  asserts  that,  provided  a  feasible  pair  \,u  exists  at  all  and  a  feasible 
quintuple  of  y's  exists  at  all,  there  must  exist  a  feasible  pair  \,u  and  a 
feasible  quintuple  of  y's  such  that  the  corresponding  u  and  v  are  equal. 
If  a  feasible  pair  \,n  and  a  feasible  quintuple  of  y's  are  found  for  which 
u  =  v,  then,  since  u  is  a  lower  bound  for  v,  v  is  minimized  for  feasible 
quintuples  of  y's.  This  latter  problem  —  to  minimize  v  for  feasible  quintuples 
of  y's  —  is  called  the  "dual"  of  our  problem  to  maximize  u  for  feasible  \,\i. 

We  now  examine  the  problem  of  finding  a  feasible  quintuple  of  x's  (i.e., 
arising  from  a  feasible  pair  \,|i)  and  a  feasible  quintuple  of  y's  such  that 

Xlyl  +  x2y2  +  *3y3  +  xkyk   +  x5y5  =  v  -  u  =  °  • 
The  sum  of  the  nonnegative  terms  x.y.  is  zero  if,  and  only  if,  each  term 

x.y.  =  0.  But  x.y.  ■  0  if,  and  only  if,  x.  =  0  or  y  =  0  (or  both).  So  we 

seek  solutions  of  the  column  and  row  systems  such  that  certain  of  the  x's  are 

zero,  the  complementary  y's  are  zero,  and  all  the  remaining  x's  and  y's  are 

nonnegative.  The  column  system  has  two  "degrees  of  freedom"  since  it  consists  of 

six  equations  in  eight  unknowns,  and  the  row  system  has  three  "degrees  of  freedom" 

since  it  consists  of  three  equations  in  six  unknowns.  Accordingly,  we  plan  to  set 
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tvo  of  the  x's  equal  to  zero  and  solve  for  the  three  remaining  x's,   and  set  the 
complementary  three  of  the  y's  equal  to  zero  and  solve  for  the  two  remaining  y's, 

The  number  of  possible  pairs  of  x's  (and  complementary  triples  of  y's)  is 
5J/2J  31  =  10.   In  our  example  two  of  the  ten  possibilities  fail,  namely  x,  =  0, 
x.  =  0  and  x^  =0,  x=  =  0*  because  the  resulting  systems  are  inconsistent. 
[This  corresponds  to  the  graphical  fact  that  lines  (l)  and  (k)   are  parallel  and 
lines  (2)  and  (5)  are  parallel  in  Figure  1. ]  The  eight  possible  cases  turn  out  to 
be  as  follows  (where,  for  example,  case  2k   corresponds  to  taking  xp  =  0  and 
xk   =  0). 

TABLE  1 


Case 

K 

n 

xi' 

xo> 

X3, 

XV 

X5 

u 

X- 

feasible? 

yl> 

y2' 

y3, 

yu> 

y5 

V 

y- 

feasible? 

12 

-K 

-l 

o, 

o, 

-2, 

3, 

1 

8 

no 

2, 

1, 

o, 

o, 

0 

8 

yes 

13 

o, 

1 

o, 

2, 

o, 

3, 

-1 

6 

no 

1, 

o, 

1, 

o, 

0 

6 

yes 

15 

-2, 

0 

0, 

1, 

-1, 

3, 

0 

7 

no 

2, 

o, 

o, 

o, 

-1 

7 

no 

23 

-2, 

1 

2, 

o, 

o, 

1, 

l 

k 

yes 

o, 

-1, 

2, 

o, 

0 

k 

no 

2k 

-1, 

2 

3, 

o, 

If 

o, 

l 

2 

yes 

o, 

1, 

o, 

-2, 

0 

2 

no 

Zk 

-3, 

1 

3, 

-If 

o, 

o, 

2 

3 

no 

o, 

o, 

1, 

-i, 

0 

3 

no 

35 

-1, 

1 

1, 

1, 

o, 

2, 

0 

5 

yes 

o, 

o, 

2, 

o, 

1 

5 

yes 

^5 

1* 

3 

3, 

If 

2, 

o, 

0 

1 

yes 

o, 

o, 

o, 

-2, 

-1 

1 

no 

Case  35,  where     x.   =  0,     x_  =  0     and     y.    =0,     y2  =  0,     y^  =  0,     yields  non- 
negative  values  for  the  remaining     x's     and    y's,     namely,     x.    =1,     x?  =  ^f 
x.    =  2     and    y,  =2,     y=  =  1«     These     x-     and    y-quintuples  are  both  feasible 
(the     x's     resulting  from    \  =  -1,   u  =  1)     and  so  yield  a  solution  to  our  example, 
with    u  =  5     as  the  desired  maximum  (and    v  =  5     as  the  minimum  for  the  dual 
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problem).  Notice  that  the  eight  cases  in  Table  1  correspond  to  the  eight  points 
of  intersection  12,  13,  15,  23,  2k,   3k,   35,  ^5  in  Figure  1. 

Thus  our  problem,  which  may  have  seemed  initially  to  be  an  analytic  one  (of 
"calculus  type" )  has  been  solved  by  a  systematic  finite  search,  essentially  com- 
binatorial in  nature. 

CONDENSED  SCHEMATA.  We  now  develop  an  organized  procedure  for  obtaining  the 
information  summarized  in  Table  1.  We  transform  from  the  initial  schema  to  a 
class  of  condensed  schemata,  each  of  which  exhibits  column  and  row  systems  having 
the  same   (x,u)-  and   (y,v) -solutions  as  the  column  and  row  systems  of  the 
initial  schema.  Instead  of  equating  to  zero  a  pair  of  x's  and  then  solving  for 
the  remaining  x's  and  u,  we  will  solve  the  corresponding  pair  of  column  equa- 
tions (in  the  initial  schema)  for  \,\x     and  use  these  solutions  to  eliminate  \,\i 
from  the  remaining  column  equations.  Similarly,  instead  of  equating  to  zero  the 
complementary  triple  of  y's  and  then  solving  for  the  remaining  y's  and  v,  we 
will  solve  the  first  two  row  equations  (in  the  initial  schema)  for  the  remaining 
two  y's  and  use  these  solutions  to  eliminate  these  two  y's  from  the  third  row 
equation. 

For  example,  from  the  elimination  process  indicated  by 


J-]      "*  o     *  "3      Mi     ^5 

\ 

-1       1  |    0       1-1       1 

M 

2-111-2       1-3 

1 

-2       3  i  -1       5-2       9 

=0 


\l       (Solve  I  for  yx,y2) 


^v  f  II  (Then  substitute  in  II 
to  eliminate  y,,y2) 


I  II 

(Solve  I  for  \,\x)     (Then  substitute  in  II 

to  eliminate  \,n) 
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we  obtain  equivalent  systems  of  equations  exhibited  by 

0   0   y3  yk    y^  1 

=*-y. 


1 

1 

1 

1 

-1 

0 

-2 

? 

2 

1 

1 

0 

-1 

-1 

-k 

-1 

-2 

3 

1 

8 

1  - 


=*.  =[i     =x^  =x^  =x  =u 

Now,  by  setting     x1  =  0     and    x?  =  0,     we  read  off 

\  =  -U,     n  -  -1,     x3  =  -2,     xk  =  3,     x^  =  1,     u  -  8  , 
and  by  setting    y-   =0,     y.    =  0,     y,.  =  0,     we  read  off 

yl  =  2>     y2  =  lf     v  =  8  • 
Here  we  have     y-feasibility  but  not     x-feasibility. 

The  first  two  columns  of  the  new  schema  serve  only  to  state  the  two  equations 

which  express     \     and    u     in  terms  of    x.,x_.     Deleting  these,  we  get  a  "condensed" 

(or  reduced)  schema 

(12)  y3     yk     y,.     1     yes 

=-yl 

=-y2 

=v 
no  =x_    =x.    =x_   =u 

that  exhibits  all  the  information  pertinent  to  the  key  equation,  in  which  \  and 
|i  do  not  appear. 

and 


1 

1 

-1 

0 

-2 

2 

1 

0 

-1 

-1 

-2 

3 

1 

8 

If  we  had  eliminated  \  and  n  via  xp  ana  x~ 


instead  of  via  x   and 


x_,  and  had  solved  the  first  two  equations  for  yp  and  y_  instead  of  y,   and 
y~,  we  would  have  obtained  the  (full)  schema 


0 

0 

yl 

yk  y5 

1 

? 

l 

0 

-1 

i   -i 

1 

3 

l 

1 

1 

-1       0 

-2 

-2 

1 

2 

1     1 

h 

=\   =U   =X   =X^  =X   =U 


and  the  corresponding  condensed  schema 


=-y- 


=v 
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(23)  y1    yk    y5  1 


no 


x? 

-1 

1 

-1       1 

X3 

1 

1 

-1 

0     -2 

2 

1 

1     u 

yes 

=X1 

*k 

=x_    =U 

5 

=-y. 
=-y- 


=v 


Setting 


x2  =  °' 


=0  and  y1   =  0,  y^  =  0, 


y_  =  0  we  read  off 


x1  =»  2,  Xj»  1,  x_  =  1,  u  =  h     and  y2  =  -1, 


y3  =  2,  v  =  k 


\  =  -2,  jx  -  1 

from  the  full  schema,  and  we  read  off 

2, 

from  either  the  full  or  the  condensed  schema.     Here  ve  have     x-feasibility  hut  not 
y-feasibility. 

It  is  possible,  however,   to  pass  directly  from  (12)  to  (23).     From  the 
process  indicated  by 

(12)  y3     y^     y5     1     yes 


1 

1 

-1 

0 

-2 

2 

1 

0 

-1 

-1 

-2 

3 

1 

8 

=-y] 

=-y, 


=v 


Solve  the  y  -equation  for 


r3, 


and  substitute  in  the 


no  =x^  =Xi.  =x,_  =u 


^3  "AU  -5 
Solve  the  x~ -equation  for 
x.. ,  and  substitute  in  the 
three  remaining  equations 
to  eliminate  x. 


two  remaining  equations  to 
eliminate  y0 
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ve  obtain  the  equivalent  systems  of  equations  summarized  by 


(23 

)r1 

yk 

y5 

l 

no 

*3 

1 

-i 

0 

-2 

=-y 

X2 

-1 

i 

-l 

1 

=-y 

1 

2 

i 

l 

I| 

=sv 

yes  =xx  =x^  =x5  =u 

In  the  transition  from  (12)  to  (23)  the  marginal  labels  x,,x-  and  -y^y, 
are  replaced  by  x_,x,  and  -y^y-,,  and  all  remaining  marginal  labels  are  un- 
changed, while  the  interior  entries  are  transformed  appropriately.   This  is  a 
particular  instance  of  an  important  but  simple  operation  called  a  "pivot  step;" 
which  can  be  described  more  generally  as  follows: 


x 


•  •     3-     •••      D     * 


•  •     v-     •••     Ci     • 


=-y 


=-y 


•  •   • 

y 

T\ 

» 
• 

: 

1* 

•  •   • 

• 

-i 

a 

• 

•   •   • 

A   ... 

X 

•   •   • 

• 

-1 
-ca 

•  •  • 

d-ca"  d 

• 
• 

• 

• 

— r\ 


=-y 


•  •  ■    •  C      •  •  •    =  r      •••  •  •  •      — Jv       •••        *~  £         ••• 

Here  the  marginal  labels     x*,  |*     and     -y*,Tj*     are  replaced  by     |*,x*     and     -r\*,y* 
and  all  other  labels  (such  as     x,  g     and     -y, tj)     are  unchanged.     This  interchange  of 
starred  marginal  labels  only  signalizes  the  fact  that  the  transition  from  the  gen- 
eral schema  on  the  left  to  that  on  the  right  is  accomplished  by  solving  the     (*- 
equation  (column)  and  the     y*-equation  (row)  on  the  left  for     x*     and     tj*, 
requiring     a  ^  0,     and  then  substituting  for     x*     and     r\*     in  all  the  remaining 
column  and  row  equations  to  eliminate     x*     and     i\*     from  them.     The  formal  rules 
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for  the  trans format ion  of  interior  entries  in  the  above  pivot  step  may  be  sum- 
marized as  follows:  Replace  the  pivot  a(^  0)  by  l/a;  multiply  each  remaining 
entry  b  in  a's  row  by  l/a  and  each  remaining  entry  c  in  a's  column  by 
-l/aj  add  to  every  other  entry  d  the  product  (-c/a)b  of  the  new  entry  -c/a 
in  dfs  row  and  a's  column  and  the  old  entry  b  in  d's  column  and  a's  row. 

The  entire  class  of  condensed  schemata  for  our  illustrative  program  is  ex- 
hibited in  Table  2.  There  are  eight  schemata  corresponding  to  the  eight  cases  in 
Table  1,  as  well  as  to  the  eight  points  of  intersection  in  Figure  1.  In  the  net- 
work of  eight  nodes  (points)  and  eighteen  branches  (lines),  each  node  corresponds 
to  the  immediately  adjacent  schema  and  each  branch  joining  two  nodes  to  the 
existence  of  a  pivot  step  (in  either  direction)  from  one  to  the  other  of  the  two 
corresponding  schemata  (with  appropriate  rearrangement,  if  necessary,  of  rows 
and/or  columns). 

SIMPLEX  METHOD.  The  network  of  schemata  for  a  general  problem  in  m+n 
variables  x  and  m+n  variables  y,  with  m  "degrees  of  freedom"  in  the  column 
system  of  equations  and  n  "degrees  of  freedom"  in  the  row  system,  may  have  as 

many  as     (m+n)l/mjnj     nodes   (and     (m+n)j/2(m-l).' (n-l)j     branches).     For  example, 

ho 
if    m  =  50     and    n  » 15Q,     the  number  of  nodes  may  be  the  order  of     10     .     So,   for 

a  large-scale  problem,   it  is  clearly  impracticable  to  construct  the  whole  network 

of  schemata.     Fortunately  the  Simplex  Method  (19V7)  of  G.  B.   Dantzig  provides  an 

effective  means  of  getting  to  an  optimum  by  constructing  only  a  very  small  part 

of  the  network. 

Starting  from  a  schema,   or  node,  which    fe     y-feasible  (or     x-feasible),  the 

Simplex  Method  proceeds  through  a  succession  of  pivot  steps,  or  branches,  usually 

not  more  than    m+n     in  number,   each  of  which  preserves     y-feasibility  and  makes     v 
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TABLE  2.  Network  of  equivalent  schemata 
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smaller  or  at  least  not  larger  (or  preserves     x-feasibility  and  makes     u     larger 
or  at  least  not  smaller)  until  a  terminal  schema  is  attained  that  exhibits  both 
x-     and    y-feasibility  or  proves  such  joint  feasibility  cannot  exist.     For  example, 
in  Table  2,   starting  from  the     y-feasible  schema     12     with     v  =  8,     it  is  possible 
to  pivot  to  the     y-feasible  schema     13     with     v  ■  6     and  then  to  the     y-feasible 
schema     35    with     v  =  5,     where     x-feasibility  is  reached  also  and  optimality  thus 
secured.     Alternatively,   starting  from  the     x-feasible  schema     U5     with     u  =  1, 
it  is  possible  to  pivot  to  the     x-feasible  schema    2U    with    u  =  2,     then  to  the 
x-feasible  schema     23     with     u  =  h,     and  finally  to  the     x-feasible  schema     35 
with     u  ■  5,     where     y-feasibility  is  reached  also  and  optimality  thus  secured; 
at  the  same  time  it  is  possible  to  pivot  directly  from     U5     to     35.       The  double 
lines  in  Table  2  indicate  the  possibilities  in  our  illustrative  program  for  pivot 
steps  preserving     y-feasibility  and  decreasing     v     or  preserving     x-feasibility 
and  increasing    u.     The  single  and  double  arrows  indicate  directions  of  increasing 
u     or  decreasing     v,     respectively;  note  that  all  these  directions  lead  towards 
the  node     35,     at  which  optimality  is  secured. 

CONCLUSION.     Until  quite  recently  applications  of  mathematics  occurred 
chiefly  in  the  physical  sciences  and  engineering,   and  drew  mainly  on  calculus, 
differential  equations  and  other  parts  of  analysis.     Now  both  the  fields  of  appli- 
cation and  the  kinds  of  mathematics  needed  and  used  are  vastly  broader.     Many 
advanced  in  theory  have  come  closer  to  their  use  in  applications,  both  in  .  time 
and  in  substance  —  especially  in  the  newer  areas. 

Thus,   linear  programming  which  started  in  19^7  is  already  receiving  wide  usage 
in  physical  and  social  sciences,   engineering,  business  and  government  operations. 
Today  many  people  who  do  not  consider  themselves  mathematicians  need  to  understand 
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the  concepts  and  methods  of  linear  programming  in  the  regular  course  of  their  work. 
Conversely,  the  unsatisfied  needs  in  applications,  for  both  methods  and  methodology, 
have  given  direct  stimulus  to  theoretical  research. 

This  paper  has  included  a  modest  sample  of  a  newly  developing  "combinatorial 
linear  algebra."  At  the  start  our  illustrative  example  seemed  to  be  an  analytic 
problem  which  might  be  treated  by  methods  like  standard  ones  of  calculus.  By 
focusing  attention  on  the  "key  equation,"  however,  and  recognizing  that  a  solution 
(if  any  exists)  must  arise  within  a  finite  class  of  simple  possibilities,  we  were 
led  to  view  our  problem  as  a  combinatorial  one  pertaining  to  a  finite  network  of 
"condensed  schemata"  as  nodes  and  "pivot  steps"  as  branches,  within  which  the 
Dantzig  "simplex"  algorithm  determines  a  remarkably  short  path  to  the  desired 
optimum.  Similar  combinatorial  structure  can  be  identified  in  other  linear 
"problems  of  organized  complexity."  It  seems  likely  that,  by  the  natural  mathe- 
matical process  of  abstraction  and  generalization  from  various  problems  with 
common  features,  we  will  soon  have  a  recognized  body  of  "combinatorial  (methods 
in)  linear  algebra"  —  of  which  some  elements  have  been  set  forth  by  the  author  [1], 

Although  the  ideas  are  new,  they  are  not  difficult.  Only  the  simpler  tech- 
niques of  systems  of  linear  equations,  matrix  algebra,  and  multidimensional 
analytic  geometry  are  needed  for  introducing  linear  programming,  as  here  portrayed, 
and  for  other  salient  topics  in  applied  combinatorial  mathematics.  But  while  not 
difficult,  the  ideas  can  be  far-reaching.  For  example,  all  the  known  theory  con- 
cerning systems  of  linear  inequalities  can  be  developed  in  a  thoroughly  constructive 
fashion  by  an  organized  procedure  (of  combinatorial  linear  algebra)  based  on  the 
quite  elementary  process  of  "Gauss-Jordan  elimination." 


COMBINATORIAL  ANALYSIS  OF  A  2-BY-3  MATRIX  GAME 
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v-u  =  x1y1+x2y2+x3y3+xuyu+x5y5 
holds  for  any  solutions  of  the  column  and  row  equations 


3     —U     —5 


in  schema  at  left;     v  >  u     if  such  x's   and  y's  are  non- 
negative   (called  "feasible");     v  =  u     when  vcn  Neumann's 
"minimax"   is  attained.     The  following  nine  schemata 
exhibit  equivalent  systems  of  column  and  row  equations 
(in  "canonical"   form)  frcm  which  "basic"   solutions  can 
a  0     the  two  x's  at  left  and  three  y's  at  top  in  each  case. 


i  :=-y, 
4^ 


be  read  by  setting 

Underlined  (or  overlined)  corner  labels  connote  basic  x's   (or  y's)     >  0.     Schema  i+5 

yields  the  desired  minimax. 
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The  possible  computational  "pivot  steps"   from 
one  schema  above  to  another  are  here  diagram- 
med.    The  highly  effective  Simplex  Method 
(I9U7)  of  G.   B.   Dantzig  starts  frcm  a     y- 
feasible  schema  (such  as  IJ),   pivots  to 
another     y-feasible  schema  at  which     v     is 
less  or  at  least  not  greater  (such  as  ll+  or 
35)>   and  so  on,   until     x-feasibility  is  also 
attained  (here  at  E5).      Alternatively,   the 
Simplex  Method  can  move  through     x-feasible 
schemata  increasing     u     or  at  least  not  de- 
creasing    u  (such  as  23  -»3^  -♦51)  until     y- 
feasibility  is  also  attained  (here  at  ^5). 
[Note:     In  Dantzig' s  terminology,   the     y- 
program  and     y-feasible  Simplex  Method  are 
"primal"  while  the     x-program  and     x-feasible 
Simplex  Method  are  "dual."] 


GRAPHICAL  AND  COMBINATORIAL  ANALYSIS  OF  A  MINIATURE  LINEAR  PROGRAM 

Maximize        3\+2p. 

A\\i  constrained  by 


(0) 


-  \-   H+  2  >  0 

(1) 

\+  n-  1  >  0 

(2) 

\+?n-  2  >  0 

(3) 

-  \       +  1  >  0 

(M 

-6\-5u+io  >  o 

(5) 

yU 


\ 

-1 

1 

1 

-1 

-  6 

3 

=0 

n 

-1 

1 

2 

0 

-  5 

2 

=0 

1 

2 

-1 

-2 

1 

10 

0 

=v 

=x„ 


=x2  =x3 


=x. 


=x5  « 


These  column  and  row  equations  imply 
the  key  identity 

v-u  =  ^yi+x^2+x3y3+a%y4+x5y5  ' 


Note.  Lines  (l)  and  (2) 

(2)"*  (5)\  (D\  \(3)  are*arallel- 

Graphically,  each  constraint  inequality  (at  upper  right)  determines  a  halfplane,  as 
slotted  above;  the  five  constraints  together  determine  the  shaded  "feasible"  region, 
[he  "objective"  function  (0)  has  contour  (or  level)  lines  parallel  to  the  broken  line 
shown,  with  values  increasing  in  direction  of  the  arrow.  The  desired  maximum  appears 
-,o  occur  at  the  point  U5  (\=1,  n=V5)  ™here  3\+2p.  =  23/5. 

jomblnatorially,  we  deal  with  x's  and  u  defined  (in  terms  of  \,u)  by  the  six  column 
vquations  of  the  schema  above  at  right,  and  also  with  y's  and  v  satisfying  the 
-chema's  three  row  equations.  If  such  x's  and  y's  are  also  nonnegative  (called 
feasible"),  then  by  the  key  identity  above  at  right  the  corresponding  u  and  v  are 

u  <  the  desired  maximum  of  the  objective  function  (0)  <  v  . 

i?o  trap  the  desired  maximum  we  seek  feasible  x's  and  y's  giving  u  =  v,  which  requires 
K£«0  or  y^=0  (or  both)  in  each  term  x.^  in  the  key  identity.  Trying  two  x's=0  and 
;he  complementary  three  y's=0,  we  are  led  to  the  nine  "basic"  solutions  tabulated 
jelov,  which  correspond  to  the  nine  points  of  intersection  in  above  graph  (including 
?5  obtained  by  extending  lines  2  and  5  to  intersect  at  \=5,  H52-1*). 
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Ihese  basic  solutions  can  be  conveniently  computed  by  transforming 
:hrough  "pivot  steps")  to  nine  schemata,  displayed  on  the  other  page, 
.^hich  contain  equivalent  systems  of  column  equations  in  the  x's  and  u  and 
of  row  equations  in  the  y's  and  v.  If  desired,  \   and  \x   can  be  included; 
e.g.,  the  block  at  right  goes  with  schema  13  on  the  other  page. 
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DUAL  PAIRS  OF  LINEAR  PF 
AND 
SIMPLEX  MET 

1.  Introduction. 

These  notes  are  intended  to  summarizi  i 

pertaining  to  dual  pairs  of  linear  program  -lay  k«.  ,    rol 

constructive  computational  algorithms  for  obtaining  solu    Lo  The 

simplex  method  (1947)  of  G.    B.    Dantzig  [2],    and  its  variants,    are  the 
basic  computational  techniques,    and  some  of  these  are  developed  be- 
low.     The   simplex  method  is,    however,    much  more  than  the  basic 
computational  tool  of  linear  programming.      It  is  also  a  combinatorial 
algorithm  that  provides  constructive  means  of  establishing  funda- 
mental theorems  of  linear  programming,    matrix  games,    and  linear 
inequalities  (see  [5]  for  a  discussion  of  these  points). 

Familiarity  with  elementary  matrix  manipulation  is  assumed. 
Simple  examples  illustrating  the  three  algorithms  presented  are 
found  at  the  end  of  the  respective  sections. 

2.  Dual  Linear  Programs  and  Pivot  Steps. 

A  pair  of  dual  linear  programs  (in  "canonical  form") 
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Primal  (Row)  Program 


Minimize 
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o  11 


constrained  by 
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x    >  0  , 
1  — 


and 


Dual  (Column)  Program 


x     >    0 
n  = 


const  i 

-X        --     B     r     A  A       <     0     , 


X     >    0 


Maximize 


u     =    d  +    u      ,b,  + 
o  n+1   1 


constrained  by 
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n+m  = 


Maximize 


u     =  d  +  U    B 
o 

constrained  by 

U     =C  +  UA>    0     , 


is  conveniently  exhibited  in  the  schema 


U     >    0 
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In  the  dual  linear  programs  (Z.  1),    (Z.  Z)     or  their   schema 

Z 
(Z.  3),    x       and    X     =  (x    ,,,...,  x          ),     the  basic  variables  of  the 
o  n+1  n+m  

primal  program,    are  expressed  in  terms  of    X    =  (x  ,  .  .  .  ,  x   )   ,     the 

1  n 

non-basic  variables   ;    similarly,     u       and    U     =  (un,  .  .  .  ,  u   )   ,     the 
o  1  n 

basic  variables  of  the  dual  program,    are  expressed  in  terms  of 


U     =  (u      n,  .  .  .  ,  u          )   ,     the  non-basic  variables, 
n+1  n+m  


A  pivot  step  on  (Z.  1),    (Z.  Z)  or  (Z.  3)     with  pivot  entry 

a.,  -f  0     is  a  Gauss-Jordan  or  complete  elimination  step  which  simu- 

ltaneously  solves  the     i       (row)  equation  of  the  primal  for     x.     and 

the    j         (column)  equation  of  the  dual  for    u      .,     and  uses  these 

equations  to  eliminate  x.     and    u      .     from  the  remaining  row  and 

J  n+i 


column  equations  at  the  cost  of  introducing     x  and    u.  .       x.     and 

n+i  j  j 
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uT,  +  i     thereby  become  basic  variables  and    x  and     u.     non-basi< 


n-li 


J 


variables.      The  pivot  step  with  pivot  entry     a  =  a      (/  0)     (*  denotes  U,< 
pivot  entry)     takes  the  schema 
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the  other  marginal  variables  and  labels  remaining  in  the  same 
positions.      Thus,    successive  schemata  obtained  by  pivot  steps  simply 
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re-express  the  original  pair  of  dual  linear  programs  through  different 
partitions  into  sets  of  basic   (or  dependent)  and  non-basic   (or  mdepenc 
variables.     Any  such  schema  or  representation  has  the  form 


u 


\ 

1 

n 

1 

_i  ili_  _  i 

c 

XL- 

Un+1 

bi '  an 

ain 

n+m 

b       .a            ... 
m  1      ml 

a 
mn 

=u  =u, 

o  1 


=u 


n 


x 


=     -X 


n+1 


>      - 


=   -x 


n+m 


U 


i     x1 

B  1  A 
1 

=    X 

°-Z 

=   -X 

:v^\ 


where  the  barred  variables  are  a  rearrangement  of  the  original  vari- 
ables and  the  barred  entries  are  determined  by  the   succession  of  pre- 
ceding pivot  steps.      The  pair  of  dual  linear  programs  (Z.  6)     are  equi- 
valent to  those  of  (2.  3)     in  the  sense  that  the  row  programs  of  (Z.  3) 
and  (Z.6)  have  the  same  solutions,    and  the  column  programs  of  (Z.  3) 
and  (Z.6)  also  have  the  same  solutions.      Basic   solutions  to  both 

programs  are  associated  with  any  schema  (Z.  6);    they  obtain  by 

-1  -Z 

setting  the  non-basic  variables     (X     and    U  )     equal  to  zero,    thereby 

determining  values  for  the  basic  variables     x         =   -b,  .  .  .  ,    x 

°  n+1  1  n+m 

=  -b        (X    =   -B  ),     x     =  d  =  u     ,   u    =  c.,  .  .  .  ,   u     =  c     (U   =  C  )  . 

m  o  oil  n         n 
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If    -b     >  0,  .  .  .  ,    -b       >  0    (-B  >  0)    a  basic  feasible  primal  solution 
obtains;    if    c    >    0,  .  .  .  ,    c     >    0     (C  <>  0)    a  basic  feasible  dual  solu- 

tion  obtains.      If  both  primal  and  dual  basic  feasible   solutions  obtain. 
then  they  constitute  optimal  solutions  to  the  programs. 

3.      Primal  and  Dual  Simplex  Methods. 

A   simplex  method  for   solving  a  pair  of  dual  linear  programs 
(2.  3)     in  canonical  form  is  a  finite  sequence  of  schemata  or  repre- 
sentations exhibiting  equivalent  pairs  of  dual  linear  programs  obtained 
by  successive  pivot  steps,    with  prescribed  pivot  entry  choice  rules, 
which  obtain  a  schema  exhibiting  optimal  solutions  to  both  programs, 
or  the  non-compatibility  of  the  primal  and/or  the  dual  constraints. 
Letting   07      denote  non-negative  entries  and       G        non-positive 
entries  all  the  possible  "terminal"  cases  can  be  exhibited  in  schema 
form  as  follows: 


(3.1) 


(optimal  solutions) 
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(3.2) 


(primal  consl  rainl  - 
compatible;    dual 
constraints  non- 
compatible) 


(3.3) 


(dual  constraints 
compatible;  primal 
constraints  non- 
compatible) 


(3.4) 


— 

1 

i 
! 



i 

4-    - 

0 

1     • 

i 

—     — 

— 

—    — 1 

1 

T 

i 

e 

i              i 
CD    .            .     ; 

(_ 

1 
1 
1 

9 

(primal  and  dual 
constraints  non- 
compatible) 


A  primal    (dual)     simplex  method    is  a  simplex  method 
beginning  with  a  primal  (dual)  basic  feasible  solution  with  pivot  steps 
which  maintain  primal  (dual)  feasibility  in  each  succeeding  schema. 
A  primal  (dual)  pivot  choice  rule  is  as  follows: 
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If  a  schema  (2.  6)  does  not  exhibit  optimal  solul  .  •  both 

programs  there  must  exist  a    c.  <  0     for   some     i     (a  b    >  0 

J  J  i 

for  some     i).      Either     (a)     every  entry  in  the  i  olumn  oi     c     <  0 

J 

is  non-positive  (every  entry  in  the  row  of    b.  >  0     is  non-negati 
or     (b)     there  exist  positive  (negative)  entries. 

(a)  The  schema  exhibits  the  non-compatibility  of  the  dual 
constraints  (of  the  primal  constraints). 

(b)  Choose  as  pivot  entry    a       >  0      (a.     <  0)     satisfying 


b.  be  c 

K  =       mav  S  I  It  =         S 


max       s_       (  £ =       max 


kj  a     >0         sj  u  ais^u        ais 

SJ 

If  an  initial  schema  does  not  exhibit  a  primal  (dual)  basic 
feasible  solution  some  special  "initiating"  device  (see  Item  6  below) 
is  introduced  (e.g.  ,    "artificial  variables")     enabling  consideration 
of  an  allied  problem  whose   solution  ("phase  1  computation")  provides 
a  primal  (dual)  basic  feasible   solution  for  the  original  problem.      The 
original  Dantzig  method  [2]  is  a  primal  simplex  method;    the  Lemke 
paper  [4]  describes  a  dual  simplex  method. 
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The  proofs  for  termination  of  a  simplex  method  in  a  fini 
number  of  pivot  steps  use  the  fact  that  any  schema  is  uniquely  del* 
mined  by  its  associated  non-basic  variables  (of  primal  or  of  dual 

program)  and  that  there  exist  at  most    (  )  -    (  )     possible  sets 

m  n 

of  non-basic  variables.      Then,    any  pivot  steps  assuring  that  no  schema 

is  ever  repeated  guarantees  finiteness.      The  finiteness  proof  for  a 

primal  (dual)   simplex  method  in  which  no  "degeneracies"  occur, 

i.  e.  ,    in  which    c.  <  0     all    j     (b.  >  0     all    i  )      in  every  schema  is  clear: 

for  each  pivot  step  strictly  decreases  (increases)  the  value  of    d 

and  thereby  an  order  is  assigned  to  the  sequence  of  schemata.      If, 

however,    degeneracies  occur   some  form  of  lexicographic  order  must 

be  introduced  to  avoid  the  possibility  of  cycling. 

Example  1. 

Consider  the  dual  pair  of  linear  programs 

Minimize  Maximize 

x     =   -3x,    -  6x_,   -  2xn  u     =   -3u      -     u 

o  1  2  3  o  4  5 

constrained  by  constrained  by 

-x,  =     -3     +  3xn     +     4x^     +  x_  <    0  u.   =   -3     +     3u       +  u_     >    0 

4  123=  1  45     = 

-x,   =     -1     +     x,     +     3x0     +  2x,<  0  u7  =   -6    +     4u       +  3u     >   0 

5  123=  2  4  b  = 

x  ,    x2>    x3  >     0  u3  =   -2    +       u4     +  2u5  >    0 

4        5  = 
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which  are  exhibited  in  the  schema 


u 


u, 


(min) 


1 

Xl 

xz 

X3 

o  ! 

-3 

-6 

-2 

=    X 

0 

~r 

-3i 

3 

4 

1 

=    -x 

1 
-1    1 

1 

1 

3'" 

2 

=     -X 

(max) 


=u         =un       =  U-,       =u0 
o  1  2  3 


Application  of  a  primal  simplex  method,    with  pivot  entries   starred, 
results  in  the  following  sequence  of  schemata. 


1 


1 


-2   /-I  2  2  !    =  x 

u    I  -5/3J    5/3  -4/3  -5/3 

1  ! 

u     ;  -1/3;  1/3"      1/3       2/3 


=U1   =ul 


:u. 


u 


u 


1        X2 

X5 

X3 

1 

-3    |      3 

3 

4 

4 

0    |    -5 

-3 

-5 

1 

-1    *     3 

1 

2 

:U  ^U.,       =  U,_  =  U_, 

o  2  5  3 


=x 


X 


■X. 


The   solutions  are 


and 


x     =     -3,     xn  =1,      x.=  0,      x_   -  0,      x     =  0,     xr  =  0 

o  12  3  4  5 


u     =   -3,    un   =0,     u_.   =  3,      u_   =4,     u^   =  0,     u_   =  3 
o  12  3  4  5 
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Example  2. 

Consider  the  dual  pair  of  linear  programs  exhibited  in 
the   schema 


(min) 


u 


u 


u 


1 

Xl 

X2 

X3 

0    ! 
i 

2 

1 

1 

4 

3    i 

-3 

-1 

0 

5 

6  1 

-4 

-3* 

-1 

6 

i 

2  , 

-1 

-2 

1 

X 


-X 


-x. 


=    -x, 


=  U  =U  =U-        =u 

o  1  2  3 


(max) 

Application  of  a  dual  simplex  method,    with  pivot  entries  starred, 
results  in  the  following  sequence  of  schemata. 


x 


1 


u 


u. 


u, 


:U  =u_ 

o  1 


:u. 


X 


] 


2! 

1 

2/3 

1/3 

2/3 

1    i 

-5/3* 

-1/3 

1/3 

i 

-2   1 

4/3 

-1/3 

1/3 

H 

5/3 

-2/3 

5/3 

12/5  i    2/5        1/5         4/5 


=  -x.      u. 


=  -x. 


u 


-X/         u, 


1 

-3/5|  -3/5 

1/5 

-1/5 

-6/51    4/5 

-3/5 

3/5 

-1     1      1 
1 

-1 

2 

=u. 


•u 


u 


X, 


=  -x. 
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The  solutions  are 

x     =  12/5,      x.,  =  3/5,      x0  =  6/5,      x,  =  0,      x^  =  0,      xc   =  0,      x,    =  1 
o  1  ^  j  4  5  b 


and 


u     =  12/5,     u.  =  0,     u7  =  0,     u_  =  4/5,     u.  =  2/5,     uc  =  l/5,     u,   =  0 
o  1  t-  ->  4  5  b 


4.      Block-Pivot  Transformations  and  the  Revised  Simplex  Method. 

The  pivot  step  described  above  can  be  generalized  to  a  trans- 
formation in  which  a  submatrix,    rather  than  a  single  entry,    takes  on 
the  role  of  a  pivot.      In  this  section  we  use  matrix  notation  through- 
out. 

Let    A        be  a  nonsingular     r    by     r      square  submatrix  of    A   , 
and    A      ,    A      ,    A  the  remaining  submatrices  of    A   .      Then  the  schema 

1  i—i  61  L^  L^ 


(2.  3)  can  be  rewritten  as 


(min) 


1        X, 


X. 


(4.1) 


Un 


U 


1 

d   i 

cl 

cz 

-    X 

o 

2 
1 

-i 
Bi< 

1 

An 

A12 

2 

2 

2 

B2< 

1 

A21 

A22 

=  -*! 

=u 
o 

< 

-A 

\ 

(max) 
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2  2 

(here  the     U      ,    -X       rows  may  be  vacuous  if    A        omits  no  row  of 


II 


.1 


.1 


A   ,      and  the     X      ,      U       columns  may  be  vacuous  if    A        omits  no 

Li  Li  11 


columns  of    A), 


-1 


Since    A        exists,    the  first    r     constraint  equations  of  the 
primal  problem  and  of  the  dual  problem 


B.+A..X      +A     X       -     -Xn        and    C.  +  U.  A„  +  U,A,.  =  U? 
1  11    1  12    2  1  1  1      11  2    21  1 

1  2 

can  be   solved  for     X      and    U     ;    and  the  solutions  used  to  eliminate 

1  2    e 

X      and    U      from  the  remaining  row  and  column  equations  at  the  cost 

2  1 

of  introducing    X      and     U     .      The  results  of  accomplishing  this  can 

be  exhibited  in  the   schema 


(min) 


X, 


X 


(4.2) 


U, 


U. 


d-ciAnBi    i 


AnBi 


ciAu 


A 


11 


B. 


-AA'B.    J      -A     A 
l\    11     1    '  l\ 


11 


-1 
C  ,  -  C  A , ,  A 
2       1     11     \c 


-1 
A11A12 

A22"A21A11A12 


=  u 


=  U 


1 


=  U 


T 


=  -X. 


=    -X 


The  pair  of  dual  linear  programs  exhibited  in  (4.  2)  are  equivalent 
to  those  of  (4.  1)  in  the  sense  that  the  row  programs  of  (4.  1)  and  (4.  2) 
have  the   same  solutions,    and  the  column  programs  of  (4.  1)  and  (4.  2) 
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also  have  the  same  solutions.      The  transformation  from  (4.  1)  to 
(4.  2)  is  called  a  block-pivot  transformation  of  order     r   ,      the  non- 
singular  square   submatrix    A        of  order     r     being  called  the  block- 
pivot.       Any  non-zero  entry    a.,     of    A     determines  a  block-pivot    A 
ij  ^  11 

of  order  one;    the  block-pivot  transformation  of  order  one  is,    then, 
the  pivot  step  with  pivot  entry    a..  =  A        . 

Note  that  the  block-pivot  transformation  of  order     r     ex- 
changes    r     of  the  variables    u    on  the  left  with     r     at  the  bottom,    and 
r     of  the  variables     x    at  the  top  with     r     at  the  right.      Such  a  block- 
pivot  transformation  can  always  be  decomposed  into  a  succession  of 
pivot  steps,    exchanging  just  one  pair  of  variables    u    and  variables     x, 
at  a  time;    conversely,    any  finite  succession  of  pivot  steps  is  summarized 
by  a  single  block-pivot  transformation.       Therefore,    at  any  stage  of  a 
simplex  method  the  current  schema  can  be  related  to  the  initial  schema 
as  (4.  2)  is  related  to  (4.  1);    that  is,    knowledge  of    A  resulting  from 

(and  summarizing)    any  sequence  of  pivot  steps  ,    together  with  know- 
ledge of  which  variables  correspond  to  the  columns  of    A         and  which 
to  the  rows  of    A         is  sufficient,    in  the  presence  of  the  original  data 
of  the  problem  (4.  1),    to  completely  specify  the  data  of  the  schema 
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(4.  2).      The  revised  simplex  method  uses  this   simple  observation  to 
limit  the  amount  of  computing  necessary  in  a  simplex  method. 

More  specifically,    a  (primal)  revised  simplex  method  [3] 
is  a  primal  simplex  method  which  exhibits  only  the  columns  of  the 
sequence  of  succeeding  schemata  which  correspond  to  the  initial  primal 
basic  variables  and  the  column  corresponding  to  the     1  ;    further  data 
in  a  schema  is  generated  as  needed  by  referring  to  the  initial  data 
(4.  1)  for  making  a  primal  simplex  method  pivot  entry  choice.      Thus, 
for  example,    only  the  following  data  would  be  exhibited  instead  of  the 
full  schema  (4.  2)  (where,    again,     A        may  omit  no  rows  of    A): 


(4.3)  U 


U. 


d-ciAn  Bi 


AnBi 
VA2iAnBi 


cAi 


A 


11 


"A21A11 


=u 


=U. 


X 


=     x 


■X 


o 
1 


=   -X 


In  (4.  3)  the  basic  primal  solution    x     =  d-C.A       B.,      X      =     -A     B       , 

o  1    11      1  1  if    1 

2  -1 

X_,  =   -Bn  +  A_nAni  Bn     ,      obtains  by  setting  the  non-basic  variables 
2  2  21     11      1 
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equal  to  zero.      In  making  a  primal  simplex  method  pivot  choice  rule 
on  (4.  3),    columns  which  have  not  been  computed  can  be  generated  one 
at  a  time,    according  to  the  formula  indicated  in  the  transformation  of 
(4.  1)  to  (4.  2),    until  some  column  with  negative  top  entry  is  chosen 
and  appended  to  (4.  3)   (or  an  already  exhibited  column  is  chosen).      If 
no  such  column  can  be  found,    then,    of  course,    an  optimal  solution  has 
already  been  found  and  is  exhibited  in  (4.  3). 

It  is  more  convenient,    however,    to  reorganize  the  computation 
for  a  revised  simplex  method  as  follows.      Instead  of  (4.  1)   specify  the 
original  data  in  extended  tableau  form 


(min) 


1  X 


1 


1 


xz 


X, 


X 


(4.4) 


1 

d      l 

S 

1 

cz     I 

0 

0 

=    X 

o 

2 

ui 

1 
Bl     1 

An 

Au 

I 

0 

2 

-xi 

2 
U2 

1 

A21 

z2 

0 

I 

2 
"X2 

=  u 
o 

^ 

< 

2 

=ui 

2 
=  U2 

(max) 
A  pivot  step  in  extended  tableau  form  is  precisely  the   same  as  in  the 
schema  except  that  the  pivot  entry  is  replaced  by  a    1    and  the  remaining 
entries  in  the  pivot  column  by  zeros.      Then,    instead  of  (4.  3),    a 
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revised  simplex  method  exhibits  a  partial  tableau  consisting  of  the 

column  corresponding  to  the     1    and  all  columns  corresponding  to  the 

2  2 

original  baxic  variables     X     ,      X         in  their  original  order,    namely 


X, 


X 


(4.5) 


d-ciAnBi 

i 
I 

\ 

-ciAn 

I 

1       0 

1 

Au  Bi 
VA2iAnB- 

\ 
1 

1 

-A     A"1 
A21A11 

1 

1      o 

1 

1     I 
1 

=  u 


=  U, 


=     X 


X, 


1 

M 


=  U. 


In  this  form  generation  of  a  missing  column  is  particularly  simple. 
Namely,    to  generate  the  column(s)  corresponding  to    X       premultiply 
its  original  column  in  (4.  4)  by 


-ciAn 


A 


11 


"A21A11 


the  matrix  corresponding  to  the  original  basic  variable  columns  and 
an  appended  first  column,    with  a    1    in  the  top  entry  and  zeros  there- 
after.     Thus  the  missing  column(s)  obtained  is 
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ciAn 


A 


11 


'A21A11 


C2"C1A11  A1Z 
-1 
A11A12 


AZZ"A21A11  A12 


Clearly,    if  the  problem  exhibited  in  (4.  1)  has  many  columns 
and  few  rows,    use  of  a  revised  simplex  method  results  in  a  consider- 
able reduction  of  computational  effort  (if  it  had  many  rows  and  few 
columns  a  "dual"  revised  simplex  method  could  be  specified).     As  will 
be   seen  below  it  is  precisely  this  reduction  which  makes  possible 
application  of  the  decomposition  principle. 


Example  3. 

Consider,    again,    the  dual  programs  of  example  1    exhibited 
in  extended  tableau  form  (4.  4) 

(min) 


1 


Xl  X2  X3  X4  X5 


1 


u 


u. 


1 
0      i 

! 

-3 

-6 

-2 

0 

► 

0 

1 

-3     i 
| 

3 

4 

1 

1 

0 

-'   i 

1 

3 

2 

0 

1 

=    X 


:u. 


:U1      =U2        =U3        =U4        =U5 


(max) 


11-19 


In  the  first  step  the  partial  tableau  is 


ii 


u. 


0 

!     o 

0 

-3 

1   1 

0 

-1 

:° 

1 

=  u 
o 

=  U4 

=U5 

-X 


So  we   search  for  a  column  of  the  original  data  with  top  entry  negative, 
temporarily  append  it  and  pivot  as  indicated,    to  get 


1 


u 


u 


X 


4 

5 

1 

-2 

0 

2 

l4 

2 

-5/3 
-1/3 

1 
0 

-4/3 
1/3 

-X. 


=  u 


=  u 


:u. 


The  missing  columns  are 
/  1         0 


X1: 


and 


*3' 


0 
1 
0 


-4/3 


\ 


1/3/ 


-6 


2 
5/3   j 
2/3  / 


! 

0 

W 
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Since  the    x  -column  has  negative  top  entry  we  temporarily  append  it 
and  pivot  as  indicated  to  get 


u 


u, 


1 

-3  1 

0 

3 

"1 

0 1 

1 

-3 

-I1 

1 

0 

1 

=ul  : 

IU4 

=U5 

The  missing  columns  are 


xr 


\ 


1  0 
0  1 
0       0 


-3 


W 


1      0      3.-6 


x    :      0      1 


■3         4         = 


0      0       1  )    \  3   / 


3 

-5 
3 


x3: 


\ 


I 


Since  all  columns  have  nonegative  top  entry  the  problem  is   solved 
with  the  primal  solution  exhibited  in  the  partial  tableau: 
x     =     -3,      x    =  1,      remaining  variables  zero. 


The  dual  problem  solution  is  read  off  from  the  partial  tableau  and 
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the  missing  columns 


u     =   -3,     u.   =0,     u     =  3,     u.  =4,     u     =  0,     u     =  3 
o  1  Z  3  4  5 


5.      Standard  and  Canonical  Forms  for  Linear  Programs. 
The   schema 


(5.1) 


(min) 


1 


'1 


N 


v. 


d      |     c2       ... 

CN 

J. 

—  —  —  — 

bl  1  all  ••• 

1    ' 

alN 

1 

[      V  !    aMl  '  '  * 
1 

amN 

- 

=    X 


> 


u         =  u, 
o  1 


N 


V 


(max) 


(min) 


1 

X 

d 

C 

B 

A 

=u        =U 
o 


pertains  to  the  dual  pair  of  linear  programs 


Primal  (Row)  Program 
(in  "standard  form") 

Minimize    x     =     CX 
o 

(5.  la)     constrained  by 

AX  =  0 


Dual  (Column)  Program 

Maximize    u     =  UB 
o 

(5.  lb)  constrained  by 

V  unconstrained 


X  >  0 


VA     * U  >    0 
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Let    A        be  a  nonsingular  square   submatrix  of    A   ,     and 
rewrite  the  schema  above  as 


(5.2) 


V 


V 


X 


X 


d  ;  c, 

C2 

"  r 

bi  i  An 

A12 

B2        A21 

A22 

-      T,1 

_   „2 

=    X 

c 

=  0 
=  0 


=  u 


A  block-pivot  transformation  with  block-pivot    A        takes     (5.  2)  into 

j  (min) 


(5.3) 


U 


V 


d-C!All    Bl 


AU    Bl 

B2-A2iAnBi 


ciAn        C2"ciAn  Ai2 


-1 

All 

-A     A"1 
A21A11 


All   A12 
A2rA21AllA12 


u 


V 


-r 


u 


T 


(max) 


o 

=  -x! 

=  0 


.1 


(5.  3)  is  equivalent  to  (5.  2)  .      If 

(5.4) 

vacuous)      and 


A22-A2iAn  Ai2  =  °-    B2-A2iAn  Bi  =  °  <or  are 


(5.5)  A^B    <    0,         C^CjA-j1  Au  >      0 
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then  optimal  (basic)   solutions  obtain  by  setting    X     =0,      U     =  0   , 

2  111 

V     =  0   ,      and  reading  off  the  values  for     U  ,    V  ,    X  ,    x       and    y 

o  o 

A  canonical  representation  of  the  dual  pair  of  linear  programs 
is  provided  by  any  schema  (5.  3)  with  (5.  4)  holding,    namely,    any  schema 


(5.6) 


1 


0 


X 


u 


V 


d-ciAiilBi!   -ciAii      cz-ciAuAu 


An\ 


A 


-1 
11 


A_1A 
All  A12 


~A21A11 


u 


=  V 


=  u 


X 


=  0 


In  order  that    A       -A     A       A         =0      it  is  necessary  and  sufficient  that 
the  order  of  the  block  pivot    A        be  equal  to  the  rank  of  the  matrix    A   , 
If  this  is  the  case  then    B     -C  A     A       =  0  also   unless    AX  =  B     is  an 
inconsistent  system  of  linear  equations.      Notice  that  if  the  rank  of    A 
equals     M  ,     the  number  of  rows  in    A   ,     then  the  rows  of    V       in  (5.  6) 


are  vacuous,    and  (5.  6)  becomes  (where     V     =  V) 


X 


(min) 


(5.7) 


U 


d-c1An1Bl  i-c^;1 

-1        T           -i 

AnBi    i       An 

C2"C1A11  A12 
A11A12 

-    X 

o 

^x1 

„l 

2 

=  u 


=  V 


=  U 


max) 
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Using  perhaps  more  familiar  language  the  basis  for  the   schema  (5.  7) 
is    A      ,     the  basis  inverse  is    A 

In  any  case,    the  rows  of  (5.  6)  corresponding  to     V.      may  be 
dropped  to  obtain  (5.  7)  with  no  loss  of  information.     Similarly,    the 
0-headed  columns  of  (5.  7)  may  be  dropped  with  no  loss  of  information 
to  obtain  a  canonical  representation  (see  (2.  3))  which  constrains  all 
the  parameter  free  information  of  (5.  1)    : 


(min) 


X 


(5.8) 


U 


(max) 


d-ClAlilBl      I        Wl'l^l* 


An\ 


All  A12 


X 


=   -X 


=  u 


=  U 


6.      Initiating  A  Primal  Simplex  Method. 

A  primal  simplex  method  can  be  applied  directly  to  a  dual 
pair  of  linear  programs  if  they  are  in  "canonical  form"   (2.  3)   (a  basic 
solution  obtains)  and    if  all    b.  <    0     (the  basic   solution  is  primal 
feasible).      Otherwise,    an  initiating  process  or  device  is  necessary. 
The  best  known  of  these  resorts  to  the  introduction  of  new  artificial 


variables  and  is  described  below. 
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Suppose  that  the  primal  program  is  given  in  "standard  form" 

and  the  problem  is  as  in  (5.  1).      We  can  assume  that  all    b    <  0     in 

i  = 

(5.  1),  for  if  some  one  or  more  b  >  0  then  the  k-row  (equation)  of 
entries  can  be  multiplied  by  -1  to  obtain  an  equivalent  schema  with 
all    b,  <  0     .      Introduce  ("artificial")  non-negative  constrained  variables 

y.,...,y         and  an  artificial  objective  function    y     =  v    +  •  •  •    +  v     (>  0) 

1  m  "  7o       'l  'm  = 

which  is  to  be  minimized  as  specified  in  the  schema 


(6.1) 


x. 


i        wl 


-2b.    I    -SaM 
i    .  ll 


v 


1 


11 


X. 


N 


'N 


-2a 


iN 


=  x 


=  y       (min) 


IN  "1 


v  b  a 

M|         M     i        Ml 

i . 


=u  =  u. 

o  1 


(max) 


<  - 


y. 


MN  j  JM 


:u. 


N 


The  part  of  the   schema  excluding  the  top  row  specifies  a  (new)  dual 
pair  of  programs  in  canonical  form  with  all    b.  <  0  ,      and  thus  a 
primal  simplex  method  can  be  used  directly  in  minimizing    y      .      This 
computation  is  usually  called  "phase  1"  of  a  simplex  method.      If  the 
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minimum  value  of    y       is  positive,    then  no  feasible  solution  to  the 

original  primal  program  of  (5.  1)  exists.      If  the  minimum  value  of    y 

o 

is  zero,    then  all     y,   =  0     and  a  feasible   solution  to  the  primal  program 

of  (5.  1)  obtains.     A  basic  feasible  solution  to  the  primal  program  of 

(5.  1)  can  then  be  found  by  pivoting  to  make  as  many  of    y  ,  .  .  .  ,  y 

non-basic  as  possible  (they  will  already  be  non-basic  if  no  degeneracies 

occur)  and  then  deleting  all  columns  corresponding  to  non-basic     y's 

and  all  rows  corresponding  to  basic  (zero  valued)    y's  .      Thus  a  schema 

in  canonical  form  obtains.      Notice  that  this  also  provides  a  constructive 

method  for  obtaining  a  schema  in  canonical  form  from  one  in  standard 

form. 

Suppose  that  a  dual  pair  of  linear  programs  in  canonical  form 

(2.  3)  does  not  have  all    b„  <  0     (the  initial  basic  solution  is  not  primal 

feasible).      Then  write  (2.  3)  in  the  following  equivalent  standard  form 

(min) 


J 


x. 


1 


X  X 

n  n+1 


x 


n+m 


1 


v. 


(6.2) 


d_  _l    *i_ 

b      I   a 
1     .      11 


n 


In 


0 

1  0 


b      |  a 

m        m        ml 


I      . 


a        ■     0 
mni 


0  1 


=  x 


=  0 


=  0 


> 


(max) 


=u         =u, 
o  1 


=u  =u 

n  n+1 


n  +  m 
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(min) 


V 


(max) 
Artificial  variables  can  now  be  introduced  and  the  "phase  1"  process 
described  above  applied. 

Notice  that  in  either  of  these  initiating  processes  the  "initial" 
basic  variables  are  the  artificial  variables     y  .      Thus,    in  using  a 
revised  simplex  method  it  is  the  columns  corresponding  to  the    y's 
that  are  recorded  in  the  partial  tableau.      Usually  (if  the  row  equations 
are  linearly  independent)  all  the    y's     become  non-basic.     Also, 
notice  that  the  top  entry  of  a    y    column  (corresponding  to  the     x 
equation)  can  be  of  any  sign,    and  yet  no  entry  from  that  column  should 
ever  be  chosen  as  pivot  after  completion  of  phase  1.      This  is  because 
the  artificial  variables    y    must  remain  at  zero  values;    or,    equivalently, 
that  in  (5.  1)  or  (6.  2)  the     v-variables  are  unrestricted  in  sign. 

It  is  possible  to  describe  a  direct  method  for  obtaining  a 
basic  feasible  primal  solution,    i.  e.  ,     a  method  which  does  not  require 
the  introduction  of  any  new  variables  but  operates  on  the  data  of  the 
problem  as  given.      Consider,    first,    the  case  of  a  dual  pair  of  programs 
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in  canonical  form  (2.  3)  in  which  some     b    >  0   .      Then  the  rows  of 

l 

(2.  3)  can  be  rearranged  according  to  the  signs  of  the     b.     and  schema 


tically  exhibited  as 

1 


(min) 


x. 


(6.3) 


1 

d 

ci       •       • 

c 
n 

=    X 

o 

Un+1 

+ 

an 

ain 

="Xn+l 

• 

+ 

• 

• 

• 

Un+k 

+ 

aki  •    • 

\n 

n+k 

• 

e 

• 

• 

• 

u 
n+m 

e 

a           . 
ml 

a 
mn 

=  -x 

n+m 

:U  =u, 

o  1 


-u 


n 


(max) 

where  instead  of  the    b.     we  have  displayed  their   signs.     In  (6.  3) 

i 

the  part  of  the  schema  below  the  double  line  specifies  a  (sub)  dual 
pair  of  linear  programs  in  canonical  form  with    b ,<    0  ,      and  thus 
a  primal  simplex  pivot  choice  rule  for  (6.  3)  can  be  used  directly  in 
(temporarily)  minimizing     -x      ,      subject  to  the  constraints  as 
specified  below  the     n+k  row.      If  the   sought  after  minimum  value  of 
is  positive,    i.  e.  ,    the  maximum  value  of    x  is  negative, 


-x 


n+k 


n+k 
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then  no  feasible   solution  to  the  primal  program  (6.  3)  exists.      If 

some   schema,    after  pivoting,    exhibits  a  positive  value  for    x 

n  +  k 

our  temporary  goal  has  been  reached.     A  new  sub-schema  of  the 
same  form  as  that  below  the  double  line  in  (6.  3)  can  now  be   specified 
which  contains  at  least  one  more  row  than  previously,    and  a  primal 
pivot  choice  rule  used  directly  on  it.      This  is  repeated  until  a  primal 
feasible  solution  to  the  whole  schema  obtains.      (This  idea  is  used  in 
[1]  to  specify  a  "neutral  primal  dual  simplex  method"  which  can  be 
applied  directly  to  any  dual  pair  of  programs  regardless  of  feasibility 
and  degeneracy). 

If  a  dual  pair  of  programs  is  given  with  the  primal  program 
in  "standard  form,  "  then  one  could  pivot  in  order  to  make  as  many 
O's     non-basic  as  possible  (see  (5.  Z)-(5.  3)),      delete  zero  headed 
columns  and  rows  with  right  hand  label    =  0    unless  inconsistencies 
were  discovered  (see  (5.  6) -(5.  8))}    and  thus  obtain  an  equivalent  schema 
exhibiting  a  dual  pair  of  programs  in  canonical  form  in  which  a  basic 
primal  solution  obtains.      The  method  above  could  then  be  applied 
directly. 

Example  4. 

Consider  the  primal  problem  of  example  2,    exhibited  in 
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(min) 


1  xl 

X2 

X3 

0   !     2 

1 

1 

1 

-1 

0 

6    [-4 

-3 

-1 

2    1    -1 

i 

-2 

1 

-X 


=  -x, 


and  suppose  we  set  out  to  solve  it  using  a  primal  simplex  method, 

thus  requiring  an  initiating  process.     A   sequence  of  schemata 

might  be : 

Phase  1 

1  x,  x^  x„  x .  x,.  x, 


0  1 

2 

1 

1 

0 

0 

0 

11  1 

-8 

-6 

0 

1 

1 

1 

h      1 

"3    | 

3* 

1 

0 

-1 

0 

0 

-6   | 

4 

3 

1 

0 

-1 

0 

-2  ( 

I 

1 

2 

-1 

0 

0 

-1 

y    (min) 


Ti 


y 


X. 


2       1    -2/3 
1 

1/3 

1 

2/3 

0 

0 

3       ,     8/3 

-10/3 

0 

-5/3 

1 

1 

-1      |     1/3 

1/3 

0 

-1/3 

0 

0 

-2     1  -4/3 

5/3 

1 

4/3 

-1 

0 

-1      '    -1/3 
! 

5/3® 

-1 

1/3 

0 

-1 



=    X 


=  y. 


y 
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4  5 


x. 


H/5    [  -3/5 

-1/5 

6/5 

3/5        0 

1/5 

=    X 

o 

4- 
-4/5,    Z/5 

Z 
-1/5 

-Z 
1/5 

-1         1 
-Z/5    0 

-1 
1/5 

=  ya 
=    -x 

-1      j      -1 

-1 

Z 

1         -1 

1 

=  -y 

-3/5  1   -1/5 
1 

3/5 

-3/5 

1/5        0 

-3/5 

=   -x. 

L 

y 


x, 


14/5    1       0 
1 

Z/5 

0 

-3/5 

3/5      -Z/5 

0        ,      1 

1 

0 

1 

0           0 

-6/5  |    0 

-3/5 

1 

Z/5 

-Z/5     3/5 

-1        |-1 

-1 

Z 

1 

-1          1 

-Z/5    |    0 

• 

4/5 

-1 

-1/5 

1/5    -4/5 

=  y. 


=  -x. 


=  -x 


=  -x. 


This  completes  phase  1;     so  we  drop  the    y-columns  and    y    -row 
and  continue  with  the  primal  feasible  schema 


x. 


14/5     | 

0 

3/5 

-2/5 

-6/5 

1 

-Z/5 

3/5 

-1 

z 

-1 

1" 

-Z/5  ' 
1 

-1 

1/5 

-4/5 

=  x 
o 

=-xl 

=  -x„ 


12/5    1 4/5 

1/5 

Z/5 

-3/5 1  -1/5 

1/5 

-3/5 

-1    I     z 

-1 

1 

-6/51     3/5 
\ 

-3/5 

4/5 

=    X 


=  -x. 


-x, 


=  -X 
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Example  5. 

Consider,    again,    the  problem  of  example  4  and  suppose  we 
set  out  to  solve  it  using  a  revised  simplex  method.      Then  the  initial 
data,    with  artificial  variables    y  ,      is  exhibited  in  the  extended 
tableau 


1   xl 

X2 

X3 

X4 

X5 

X6 

yl 

y2 

y3 

0    1     2 

| 

1 

1 

0 

0 

0 

0 

0 

0 

11     j  -8 
"3     1    3 

-6 
1 

0 
0 

1 
-1 

1 

0 

1 

0 

0 

1 " 

0 
0  " 

0 

~  6~ 

-6        4 

3 

1 

0 

-1 

0 

0 

1 

0 

-2     1    1 
1 

2 

-1 

0 

0 

-1 

0 

0 

i 

=y. 


In  phase  1  we  do  not  need  to  recompute,    at  every  step,    the     x    -row 
values  since  these  are  easily  obtained  (see  item  4)  when  needed  in 
phase  2. 

Phase  2. 


Y, 


V 


11     1     0 

I 

0 

0 

-3     |      1 

0 

0 

-6     |     0 

1 

0 

-2      1     0 
1 

0 

1 

=  y 


-Yi 


•y; 
y 
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-y. 


1 

Yl 

y2 

y3 

1    1 

1. 

2 

0 

2 

-4/5    i 

2/5 

0 

-1/5 

-1        ' 

1 

-1 

1 

-1 

< 

-1/5 

0 

3/5 

:y. 


-X, 


-y- 


-x. 


1 


y- 


o 


■ 


-6/5       I  0 

-1        '  -1 
I 

-2/5       I  0 


1 


2/5 
1 

-1/5 


1 


3/5 

-1 

4/5 


=y. 


-x. 


-x. 


This  completes  the  phase;     so  we  drop  the    y    -row,    and  compute  the 


needed  entries  of  the     x    -raw  by  the  formulae  indicated  in    ( 

o 


namely, 


(x1)(x4)(x2) 


(yx)  (y2)(y3) 


(        2       0         1     )    /0        2/5       -3/5\f(    0    -3/5  2/5) 

-1  1  -1 


0      -1/5  4/5y 


)   , 
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and  the  right-hand  entry- 


Then  we  continue  with  the  partial  tableau 


=  14/5 


xy 


14/5   I     0       -3/5     2/5 
-6/5 |    0  2/5   -3/5 


1 


■1 


-1     ■    -1 

I 
2/5 1     0      -1/5        4/5 


-x. 


-x 


y 


12/5  '-2/5 

-1/5 

0 

-3/5|  3/5 
-1   1  -1 

-6/5|  -4/5 
i 

-1/5 
1 
3/5 

0 

-1 

0 

-X, 


■x. 


-x. 


This  last  partial  tableau  corresponds  to  an  optimal  solution  since 
all  missing    x-columns  have  a  non-negative  top  entry. 
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III-l 

ECONOMICS  OF  DUALITY 

1,    The  Dual  Problem 

With  every  linear  programming  maximation  problem  it  has 
proved  useful  to  associate  a  closely  related  minimization  problem 

and  vice -versa       Such  pairs  of  problems  are  called  dual  linear    pro- 

1/ 
gramming  problems  The  analysis  of  dual  programming  problems 

has  attracted  a  great  deal  of  attention  both  among  economists  and 

mathematicians,    for  a  number  of  reasons: 

a       Duality  yields  a  number  of  powerful  theorems  which 

add  substantially  to  our  understanding  of  linear  programming 

b      Duality  analysis  has  been  very  helpful  in  the  solution  of 
programming  problems.     Indeed,    as  we  shall  see,    it  is 
frequently  easier  to  find  the  solution  of  a  programming  pro- 
blem by  first  solving  its  associated  dual  problem 

c.      The  dual  problem  turns  out  to  have  an  extremely  illumin- 
ating economic  interpretation  which,    incidentally,    shows 
that  old  fashioned  marginal  analysis  is  always  implicitly 
involved  in  the  search  for  an  optimal  solution  of  a  linear 


l/    The  basic  ideas  of  duality  theory  were  first  suggested  bv  John 
von  Neumann       The  theory  was  first  rigorously  developed  by 
David  Gale,    Harold  Kuhn  and  A      W.    Tucker. 
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programming  problem. 

Before  we  turn  to  the  economic  interpretation  of  the  dual, 
it  will  be  desirable  to  describe  the  meaning  of  duality  in  a  purely 
abstract  manner  without  any  reference  to  meaning  or  interpretation. 
At  the  end  of  this  section,    given  any  linear  programming  problem, 
the  reader  should  be  able  to  write  out  its  dual. 

Consider  the  general  linear  programming  problem  which  we 

will  call  the  primal  problem: 

Maximize       II  =  P_Q,  +  P0Q0  +  ■  •  ■    +  P   Q 

11  c    c  n    n 

Subject  to  _  _  ^  ^ 

ailQl  +  ai2QZ+  ■  •  •    +  ainQn  <    Cl 

(1)  

a     Q    +  a      Q   +  •  •  •    +  a        Q     <  C 
mil    1         n2     L  mn    n  —      m 

Q.    >  0  ,     .  .  ,     Q     >    0  . 

1  —  n  — 

Suppose  a  mischievous  gremlin  were  let  loose  on  this  linear 
programming  problem  and  decided  that  he  would  turn  everything 
he  possibly  could  on  its  head.      There  are  a  number  of  obvious  things 
he  would  think  of.      For  the  word  "maximize"  he  would  substitute 
"minimize".      For  the  symbol    >    he  would  substitute    <  .      Further- 
more,   a  nice  way  to  add  to  the  resulting  confusion  might  be  to  put 
the  capacity  figures     C   ,    C?,  .  .  .  ,  C       where  the  unit  profit  figures, 
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P.,    P_.,  .  .  .  ,P     ,      used  to  be  and  vice-versa.      For  good  measure 
12  n 

he  might  reverse  the  order  in  which  the  constants  appear  in  the 
inequalities,    or  better  yet,    he  might  rewrite  them  so  that  instead  of 
reading  across  we  would  now  read  down.      That  is,    where     a         was 
formerly  the  second  constant  in  the  first  inequality,    he  would  now 
make  it  the  first  constant  in  the  second  inequality.     In  other  words, 
by  reading  across  in  the  first  inequality  we  would  formerly  have 
found  the  constants,      a     ,    a      ,    a       ,      etc.      Now  we  would  find  those 
same  constants  not  by  reading  across  from  left  to  right,    but  by 
reading  down  from  the  first  inequality  to  the  second  to  the  third,    etc. 
Finally,    to  cap  the  confusion,    our  gremlin  would  probably  decide 
to  get  rid  of  our  original  variables,      Q  ,    Q    ,  .  .  .  ,  Q     ,    altogether  and 

i.  £*  xl 

substitute  for  them  an  entirely  new  set  of  variables,      V,,    V_,  .  .  .  ,  V 

1  1         Z  m 

Having  done  all  this  he  would  end  up  with  the  linear  programming  pro- 
blem which  may  be  written  as  follows: 

Minimize         a  =     C1V1  +  C0V0  +  •         +  C     V 

11  Z    Z  mm 

subject  to        a„V.  +  a,.V,  +  •  •  ■    +  a     .V       >    P. 
J  11    1  Zl    Z  ml    m  —       1 


a1    V.  +  a      V     +  •  •  •    +  a        V       >    P 
in    1  £n    l  mn    rn  —       n 


V>0,      V,  >  0,  .     .  f  V       >0 

1  —  Z  —  m  — 


Ill  -  4 


This  new  program  produced  by  the  gremlin  is  precisely  what  we  call 
the  dual.      Before  going  further,    let  us  recapitulate  the  characteristics 
of  the  two  problems  which  give  them  their  remarkable   symmetry: 

1)  If  the  primal  problem  involves  maximization  the  dual    in- 
volves minimization,    and  vice -versa. 

2)  If  the  primal  involves    >    signs  the  dual  involves    <    signs 
and  vice-versa. 

3)  The  profit  constants,      P.   ,      in  the  primal  problem  replace 
the  capacity  constants,      C.    ,     and  vice-versa. 

4)  In  the  constraint  inequalities  the  coefficients  which  were 
found  by  going  from  left  to  right  are  positioned  in  the  dual 
from  top  to  bottom  and  vice -versa. 

5)  A  new  set  of  variables  appears  in  the  dual. 

6)  Neglecting  the  number  of  nonnegativity  conditions,    if  there 
are     n    variables  and    m    inequalities  in  the  primal  problem, 
in  the  dual  there  will  be    m    variables  and    n    inequalities. 

Finally,    it  should  be  noted  that  if  we  were  to  let  our  gremlin 
loose  again  and  have  him  do  his  work  on  the  dual  problem,    he  would 
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end  right  back  with  the  problem  with  which  he  started.      That  is,    if 
he  were  to  take  the  dual  problem  and  subject  it  to  all  the  abuses  which 
he  had  heaped  on  our  original  linear  program  he  would  find  that  in  the 
end  he  would  have  undone  all  his  mischief       The  dual  of  the  dual  pro- 
blem is  the  original  linear  programming  problem  itself       It  follows  that 
given  such  a  pair  of  problems  it  is  entirely  arbitrary  which  of  them  is 
referred  to  as  the  primal  and  which  as  the  dual.     Each  one  of  them  is 
the  dual  of  the  other. 

These  rules  for  the  construction  of  the  dual  are  illustrated  in 
the  following  pair  of  numerical  linear  programming  problems       The 
reader  should  verify  that  these  problems  do  indeed  constitute  one 
another's  dual.     In  order  to  emphasize  that  minimization  problems  also 
have  a  dual  we  have  selected  as  our  illustrative  primal  program  the 
minimization  problem  (13)     of  Chapter  5. 


*    All  such  references  are  to  Chapter  5  in  William  J.    Baumol,  Economic 
Theory  and  Operations  Analysis,    Prentice -Hall,    Englewood  Cliffs, 
New  Jersey,    1961. 
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(2) 


Primal  Problem 

Minimize 

28M  +  400T  +  20R 
subject  to 

M  +  9T  +  0.  8R  >  30 
0.  6M  +  2T  +  0.  7R£     23 
M>0    T>0    R>0 


Dual  Problem 


Maximize 


30  V     +  23  V2 
subject  to 


V    +     06  V     <    28 

9V    +        2  V     <    400 

0.  8V    10.7  V     <    20 
V    >    0     v2  >    0 


It  will  be  observed,    in  our  illustration,    that  the  primal  problem  in- 
volves three  variables  and  two  ordinary  constraints  whereas  the  dual 
problem  involves  two  variables  and  three  constraints.     It  should  also 
be  pointed  out  that  one  feature  is  not  tampered  with  in  going  from  a 
linear  programming  problem  to  its  dual  -  the  inequalities  of  the  non- 
negativity  condition  retain  their  directions.      That  is,    both  in  the 
primal  and  in  the  dual  problem  each  variable  is  required  to  be  greater 
than  or  equal  to  zero. 

Before  proceeding  further  it  is  convenient  to  rewrite  our 
primal  and  dual  programs,  this  time  including  the  slack  variables 
so  that  the  ordinary  constraints  become  equations,      We  obtain 
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Primal  Problem 


Dual  Problem 


Maximize 


Minimize 


n  =  p1q1  +  p  o   +....  +  p  q 

11  L     L  n    n 

subject  to 


a  -  C.  V.  +  C_V,  +  •  •  •   +  C     V 

i     1  2     Z  mm 

subject  to 


(3) 


a    O,  +  a     Q     +  •  •  •    +  a     Q   +U  =C- 
11    1  12     2  In    n       1       1 


a     V+a     V     +  •  •  •    +  a     V     -L,=P, 
11     1      21     2  ml    mil 


a     ,Q,+a     0Q9+  •••••  +a      Q  +U    =c 

ml    1      mZ     2  mn   n      m      m 


a.    V.+a_    V,+  •  •   • +a        V     -L    =P 
In    1      in    2  mn    m      n       n 


Q  ,><>,...,  Q     >  0  ,  U   >  0,  .  .  .  ,U    >0      V>0,...,V     >  0,L  >  0,       .  ,  L  >  0. 

1  —  n  —  1  —  m-  1—  m—  1—  n— 

Here  the    U.      and    L.     represent,    respectively,    the  primal  and  dual 
slack  variables.      It  should  be  noted  that,    in  our  dual  program,    since 
the  constraints  are  "greater  than  or  equal  to"  inequalities  the  slack 
variables,    L.    ,     are  subtracted  from  the  left  hand  side  of  the  equations. 
To  give  a  simple  example,    in  translating  the  inequality     3Q    >  7     into 
an  equation  we  must  rewrite  it  as     3Q     -  L  =  7  ,     where,    as  usual, 


L  >  0 
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Problems  ; 


1.      Write  out  the  dual  of  the  following  problem 


Maximize     II  =  2Q,  +  6Q 


1 


subject  to 


4Q    +      Q2<    5 
3Q    +    2Q2  <   7 
Q1+     Q2<2 
Q1  >  0  ,     Q2  >    0 


answer 
Minimize     a  =  5V    +  7V     +  2V 

1  M  J 

subject  to 


4V    +  3V     +  IV     >    2 
1V1  +  2V2  +  1V3  -    6 


Vl  1  °   '      V2  -  °   '      V3  -    ° 


2.  Show  that  the  dual  of  your  answer  is  the  original  pro- 
gramming problem. 

3.  Write  out  the  dual  to  the  program  in  problem  1    in  slack 
variable  form. 


Interpretation  of  the  Dual  Problem 

It  is  convenient  to  begin  our  interpretation  of  the  dual  problem 
in  a  manner  which  seems  extremely  artificial.     But  the  reader  will 
see  the  artificiality  of  the  interpretation  disappear  rapidly  as  the 
analysis  proceeds  and  the  very  real  significance  of  the  duality  theorems 
will  make  this  abundantly  clear 

Let  our  primal  program  be  a  standard  production  problem, 
that  is,    the  problem  of  determining  the  profit  maximizing  output  levels 
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for  the  firm's  various  products,    subject  to  a  number  of  scarce  input 
(capacity)  constraint  limitations        The  fixed  inputs  of  the  company, 
C  ,    C^,  .  .     ,  C       ,      may  not  even  enter  directly  into  its  accounting  profit 
calculations,    particularly  if  the  warehouses,    factories  and  other 
facilities  which  these  symbols  represent  have  been  completely  amortized 
Nevertheless,    it  must  be  recognized  that  without  these  inputs  the  firm 
could  not  have  earned  its  profits.     Suppose,    then,    that  the  businessman 
in  question  decides  to  determine  what  portion  of  the  profit  on  each  of 
his  products  he  "owes"  to  each  of  the  inputs..     In  the  economist's  jargon, 
he  will  undertake  to  impute  all  of  the  company's  profits  to  its  scarce 
resources.      For  this  purpose,    he  will  seek  to  calculate  an  artificial 
accounting  price  or  value,    V  ,     for  each  of  his  inputs  and  to  choose  a 
magnitude  for  each    V     such  that  the  sum  of  these  computed  values  of 
the  scarce  inputs  going  into  any  of  his  products,    say  shoes,    is  high 

enough  to  account   for  his  profits  on  shoe  production.      That  is     values 

2/ 
of  the    V"s     should  be  chosen  so  that,    if  possible,  the  net  profit  from 


2/     We  will  see  that  some  commodities  which  the  firm  considers  pro- 
ducing may  end  up  yielding  negative  accounting  profits.      That  is, 
their  gross  profits  are  less  than  the  value  imputed  to  the  scarce 
resources  needed  to  produce  them.     As  the  reader  may  surmise, 
these  are  the  items  which  will  be  excluded  from  the  firm's  optimal 
product  line. 
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shoe  production  (and  the  net  profit  from  each  other  company  output) 
would  be  zero  if  each  scarce  input  C.  were  to  cost  V.  dollars  per 
unit. 

It  must  be  emphasized  that  the  zero  profit  condition  in  this 
problem  is  not  directly  related  to  the  zero  profit  requirement  for  long- 
run  equilibrium  under  perfect  competition.     In  our  imputation  analysis 
zero  profit  is  an  accounting  requirement.     If  accounting  values,    V  , 
are  proposed  which  do  not  impute  profits  completely  to  the  scarce  inputs, 
these  accounting  values  must  be  raised  until  the  unimputed  residue  has 
been  eliminated. 

We  shall  now  show  that  the  variables  of  the  dual  program,    the 
V  ,    V       and    V       in  our  numerical  problem  given  at  the  end  of  the  pre- 
ceding section  can  be  interpreted  as  the  required  accounting  values  of 
the  firm's  three  scarce  resources.     Let  us  assume  that  our  three  scarce 
resources  are,    respectively,    warehouse  space,    machine  time  and 
inspection  time,    so  that  our  original  linear  programming  problem  tells 
us  that  the  firm  has  available  to  it       5  (million)  cubic  feet  of  warehouse 
space,    7(hundred)  hours  of  machine  time  and    2(hundred)  hours  of  in- 
spection time.      V      may  then  be  described  as  the  accounting  value  to 


Ill -11 


the  firm  of  a  unit  of  warehouse  space,      V       as  the  accounting  value 

of  a  unit  of  machine  time  and    V       as  the  accounting  value  of  inspection 

time.     Suppose  we  tentatively  accept  this  interpretation. 

Let  us  now  look  at  the  first  inequality  in  the  dual  problem  and 
show  that  it  is  no  more  than  an  explicit  statement  of  our  no  accounting 
profit  requirement  for  the  first  item  in  the  firm's  product  line,     Q 
It  will  be  remembered  that  the  coefficient    4    of  the  variable    V      in 
that  inequality  had  a  very  definite  meaning  in  our  primal  problem. 
There  it  represented  the  number  of  units  of  warehouse  space  used  to 
produce  one  unit  of  output  1  .     Similarly,    the   second  coefficient,      3  . 
in  our  dual  inequality,    that  is,    the  coefficient  of    V      ,     in  our  primal 
problem  represented  the  amount  of  machine  time  needed  to  produce  a 
unit  of  output    1  .      Finally*    the  coefficient  of  the     V       in  our  dual  in- 
equality represented  the  number  of  units  of  inspection  time  needed  to 
produce  a  unit  of  output  1  .     In  sum,    the  three  coefficients,    the     4  , 
the     3     and  the  unity  represent  the  quantities  of  the  three  different 
inputs  which  go  into  a  unit  of  commodity  1  .     Now,    if  each  unit  of  ware- 
house  space  is  worth    V      dollars,    then    4    units  of  warehouse   space 
would  be  worth    4    times    V      dollars  and,    similarly,    if  each  unit  of 
machine  time  is  worth    V       dollars,    the  three  units  of  electricity 
would  be  worth     3V       dollars,    and,    finally,    the  one  unit  of  inspection 
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time  would  be  worth    1    times    V       dollars  if  each  unit  is  worth    V 
We  see,    then,    that  the  expression  on  the  left  hand  side  of  our  in- 
equality,     4V    +  3V     +  IV       has  a  straightforward  economic  inter- 
pretation,   given  the  meaning  which  has  tentatively  been  assigned  to 
our  variables.      That  sum  is  the  total  value  of  the  inputs  which  is  nec- 
essary to  produce  one  unit  of  output  of  commodity    1  . 

Only  one  more  step  is  needed  to  complete  our  interpretation 
of  this  inequality.      We  must  now  recall  what  is  signified  by  the  number 
2    on  the  right  hand  side  of  that  relationship.     Going  back  to  our 
primal  problem  once  more,    we  see  that  the     2    is  the  unit  profit  one 
obtains  by  producing  our  first  commodity.      Each  unit  of  commodity    1 
which  is  manufactured  yields    $2    to  the  firm.      Our  first  inequality 
can  now  be  read  to  state  the  following:    the  value  of  the  inputs  going 
into  the  production  of  a  unit  of  commodity    1    must  be  greater  than  or 
equal  to  the  profit  which  the  firm  makes  by  producing  a  unit  of  com- 
modity 1  .       Our  first  inequality,    then,    states  that  we  must  assign  to 
each  of  the  inputs  a  value  sufficiently  great  to  impute  to  them  all  the 
profits  of  output  number  1  .     Just  as  the  first  inequality  of  the  dual 
program  serves  to  impute  the  profits  of  commodity  1,    so  the  second 
inequality  imputes  all  the  profit  of  each  unit  of  output    2    to  the  company's 
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scarce  inputs.     It  states  that  the  values  of  the  three  inputs  which  are 
used  to  make  a  unit  of  commodity    2    must  account  fully  for  the  $6 
of  profit  which  are  yielded  by  a  unit  of  commodity    2  . 

We  will  see  presently  why  it  is  convenient  to  write  these 
constraints  as  inequalities  rather  than  equations;    that  is,    why  we  do 
not  directly  require  that  that  values  of  the  inputs  be  exactly  equal  to 
the  profits.     However,  one  simple  reason  can  be  indicated  now       The 
need  for  inequalities  in  the  constraints  follows  directly  from  the  fact 
that  the  number  of  variables  and  the  number  of  ordinary  constraints 
in  a  programming  problem  need  not  be  the  same.     In  our  dual  problem 
we  have,  three  variables  and  two  inequalities.      We  could  just  as  easily 
have  had,    says    six  variables  and  fifteen  inequalities       If  we  had  attempted 
to  write  those  fifteen  constraints  as  equations  rather  than  equalities,    we 
would  have  had  a  system  involving  fifteen  equations  in  six  unknowns 
and  obviously  this  is  likely  to  run  us  into  difficulties;    for  usually  it  is 
impossible  to  satisfy  a  system  of  equations  containing  more  equations 
than  unknowns.     Since3    therefore;,    in  such  a  situation  we  may  be  forced 
to  relinquish  equality  in  part  of  the  system  we  have  chosen  between 
the  two  apparently  less  desirable  alternatives,    over  imputation  and 
underimputation,      and  decided  to  favor  the  former.     That  is,    we  have 
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said,      if  it  is  absolutely  necessary  to  assign  a  value  to  the  inputs 
employed  which  is  either  greater  than  or  less  than  profit,    let  us 
assign  a  total  value  which  is  greater  than  the  profit. 

But  once  we  have  stated  the  inequalities  in  this  way,    it  may 
appear  that  there  is  no  problem  at  all  in  solving  our  linear  program. 
We  need  just  assign  values  as  capriciously  high  as  we  wish  to  each  of 
the  inputs  and  we  can  be  sure  that  they  will  more  than  account  for  all 
of  the  profits.      What  prevents  this  sort  of  arbitrary  solution  is  that 
our  dual  problem  requires  us  to  minimize      a  =  5V    +  7V     +  ZV       . 
The  value  of  the  dual  objective  function^     a  ,     also  has  an  economic 
interpretation  which  follows  directly  from  our  preceding  discussion. 
It  will  be  recalled  that  the  company  has     5  units  of  warehouse  space 
in  its  possession  so  that  the  total  value  of  the  warehouse   space  avail- 
able to  the  firm  will  be     5V      and  similarly,    the  total  value  of  the 
machine  time     available  to  the  firm  will  be     7V       because  there  are 
7  units  of  machine  time  at  the  firm's  command,    and  so  on.      Hence 
a  -  5V    +  7V     +  2V         represents  the  total  value  of  all  of  the  inputs 
which  the  firm  has  under  its  control.      To  summarize,    then,    the 


dual  problem  requires  us  to  find  the  very  smallest  valuation  of  the 
company's  stock  of  inputs  which  completely  accounts  for  all  of  the 
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profits  of  each  of  the  outputs. 

We  may  conclude  this  preliminary  discussion  of  the  inter- 
pretation of  the  dual  problem  by  ascribing  an  economic  meaning  to 

the  dual  slack  variables,     L  ,  .  .  .  ,  L      .      From  our  dual  problem  in  (3) 

in 

we   see  that,    for  example,      L,      is  given  by 

1  11    1         Zl    Z  ml  m        1 

But    P      is  the  profit  per  unit  of  input  1  ,      while,      as  we  have  just  seen, 
the  expression  in  parenthesis  is  the  accounting  value  of  the  resources 
used  in  producing  a  unit  of  output    1    (since    a        is  the  amount  of  input 
1    used  in  a  unit  of    output    1    so  that    a    V      is  the  value  of  the  amount 
of    input    1    used  in  the  production  of  output    1    etc.  )   .      We  can, therefore, 
rewrite  the  last  equation  as 

L       =     (value  of  resources  going  into  a  unit  of  output  1) 

-(the  unit  profit  of  output  1  )   . 
Thus,    we  may  consider     L      to  represent  a  sort  of  relative  loss  figure 
for  product  1  -  for  if    L      is  positive  it  tells  us  that  the  resources  used 
in  producing  output    1    are  worth  more  than  the  profits  yielded  by  that 
commodity.      We  shall  presently  return  to  these  dual  slack  figures  and 
see  precisely  what  is  meant  by  the  "loss"  which  they  represent. 
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Let  us  now  review  the  interpretation  we  have  given  the 
variables  in  our  pair  of  programming  problems.     Our  primal  and 
dual  problems  involve  the  following  four  types  of  variable: 

Q.  the  (quantity  of)  output  of  product    j  (the  primal  ordinary  variables) 

U.  the  unused  capacity  of  input    1    (the  primal  slack  variables) 

V.  the  value  (accounting  price)  of  input    i    (the  dual  ordinary  variables) 

L.  the  accounting  loss    per  unit  of  output    j  (the  dual  slack  variables)  . 

J 

Primal  variable    Q.    and  dual  variable    L.   ,    then,    both  refer 

J  J 

to  outputs  while  dual  variable     V      and  primal  variable     U      both  refer 
i 1  1 

to  inputs.       This  information  is  summarized  schematically  in  the 
following  table : 


Primal  Variable  Dual  Variable 


(physical  quantities)  (in  money  units) 

Refers  to  outputs  Q.  L. 

J  J 

Refers  to  inputs  U.  V 

The  interpretation  of  these  four  types  of  variable  must  be 
understood  clearly  before  the  reader  can  hope  to  master  the  economic 
implications  of  the  dual.     It  should  also  be  noted  that  the  primal 
variables    Q      and    U      refer,    respectively,    to  physical  output  and 
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input  quantities  and  so  must  be  measured  in  units  such  as  tons,    square 

feet,    kilowatt  hours,    etc.      On  the  other  hand,    both  dual  variables,      V 

i 

and    L.     refer  to  pecuniary  magnitudes  and  can  be  measured  in  a 
monetary  unit  such  as  the  dollar. 

So  far  the  reader  may  well  have  the  feeling  that  our  interpre- 
tation of  the  dual  program  is  rather  strained.      We  have  forced  an  odd 
sort  of  economic  meaning  on  to  our  dual  but  there  appears  to  be  little 
reason  for  an  economist  or  anyone  else  to  be  interested.     However,    the 
mathematicians  have  proved  a  number  of  theorems  about  the  dual  pro- 
blem which  dramatically  breath  life  and  power  into  the  entire  construct. 

Some  Duality  Theorems 

In  this  section  several  of  the  most  pertinent  theorems  of 
duality  theory  will  be  described  rather  briefly.      They  and  their  deriva- 
tions are  discussed  in  somewhat  greater  detail  in  the  appendix  to  this 
chapter. 

Asa  preliminary  matter,    we  may  mention  the  following  rather 
useful  theorem.     Suppose  we  have  found  any  feasible  solution  to  the 
primal  problem  and  this  solution  yields  as  the  value  of  the  objective 
function  (the  total  profit  figure)  the  number     II      .     Suppose,    moreover, 
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that  we  have  found  some  feasible  solution  to  the  dual  problem  which 
yields  the  number     a       for  the  value  of  the  dual  objective  function. 
Then    II       will  never  exceed    a     ,     i.e.,    we  will  have     IT     <    a 
This  theorem  tells  us,    in  effect,    that  tne  company's  total  profit,    II 
will  never  exceed    a  ,     the  accounting  value  assigned  to  the  company's 
scarce  inputs.     Actually  that  result  is  hardly  surprising  since,    as  we 
saw  in  the  last  section,    in  constructing  the  dual  problem  we  had  cone 
it  in  a  way  which  may  well  overimpute  company  profits  but  will  certainly 
leave  no  part  of  company  profits  unimputed  to  the  scarce  inputs.      Some- 
what less  obvious  is 

Duality  Theorem  I.        The  maximum  value  of    IT     will  be  exactly  equal 
to  the  minimum  value  of    a  . 

This  theorem  states  that  despite  our  apparent  willingness  to 
over -impute  profits  if  necessary,    everything  comes  out  well  in  the 
end!       When  optimal  solutions    have  been  found  for  the  primal  and  dual 
problems  we  will  have  assigned  a  total  value,      a  ,     to  the  company's 
scarce  resources  which  is  exactly  equal  to  total  profit,      IT  !    As  an 
example,    it  will  be  recalled  that  in  the  illustrative  problem  of  sections 
6-10,    in  Chapter  5,    maximum  profit  was  equal  to  8750   ,    as  was  shown 
in  matrix  (12),    Theorem  I  then  tells  us  that  the  minimum    value  of  the 
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objective  function  of  the  dual  problem  will  also  be     a  =  8750. 

Another  basic  duality  result  is  described  by  the  next  proposi- 
tion: 

Duality  Theorem  II.       In  a  pair  of  optimal  solutions,    the  firm  will 
produce  only  commodities  whose  accounting  loss  figures,    L  ,     are 
zero.      In  addition,    in  such  optimal  solutions,    only  inputs  which  are  used 
to  capacity  will  receive  a  nonzero  accounting  valuation,    V  .     Moreover, 
any  pair  of  feasible  solutions  to  the  primal  and  dual  problems  which 
satisfy  this  requirement  must  be  optimal! 

Symbolically,    this  theorem  may  be  written: 

in  an  optimal  solution            Q.L.  =  0       for  each  commodity    j   , 
J    J  


and                                                                           U.V     =  0       for  each  output    l 
l    l 


To  see  the  connection  between  the  symbolic  and  the  verbal  statements 
of  Theorem  II,    note  that,      e.g.  ,    the  equation    CLL.  =  0     requires 

either  that    Q    =  0     or  that    L    =  0  (  or  both).      This  means  that  if 

1  I 

Q  >  0    (if  output  of  commodity    1    is  positive)  then  we  must  have    L    =  0 
(its  accounting  loss  must  be  zero).     Similarly,    if    L    >  0     (commodity  1 
involves  an  accounting  loss)     then  we  must  have    Q    =  0   .      The  inter- 
pretation of  the  other  equations  in  Theorem  II  is  similar  and  is  left  to 
the  reader. 
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Theorem  II,    then, is  reassuring.     It  tells  us  that  the  dual 
values  make   some  economic  sense   -  inputs  which  are  in  excess  supply 
to  the  firm  are  given  a  zero  accounting  value,      V  ,      and  commodities 
which  are  associated  with  a  nonzero  accounting  loss  are  those  which, 
optimally,    should  not  be  produced. 

Looked  at  the  other  way,    the  theorem  tells  us  that  from  the 
optimal  solution  of  the  dual  problem  we  can  immediately  make  a  number 
of  predictions  about  the  solution  to  the  primal  -  we  can  be   sure,    in 
advance,    which  outputs  will  not  be  produced  and  which  inputs  will  be 
used  to  capacity.      For  example,    as  will  be  shown  later  in  this  chapter, 
the  solution  to  the  dual  of  the  illustrative  linear  programming  problem 

(31 )  of  sections  6-10,    in  Chapter  5,    is    L=0,L=0,V     =  —  , 

3 

V,    =   — —    .      With  the  aid  of  Theorem  II,    these  figures  tell  us  to  expect 
b         4  to 

both  outputs     1    and    2    to  be  produced,    and  that  both  inputs  will  be  used 

to  capacity.     Indeed^    this  turns  out  to  be  the  case,    for  it  will  be  recalled 

that  the  solution  to  the  primal  problem  is  given  by  matrix  (12)  of  Chapter 

5  and  is    Q,  =  500,    Q0  =  3750,    U     =  0,    U     =  0  . 
12  a  b 

We  may  also  observe,    specifically,    that  all  the  equations  of 
Theorem  II  are  satisfied  by  the  pair  of  solutions  just  given,    for  we  have 
Q.L.  =  (500)(0)  =  0,     Q7L7  =  (3750)(0)  =  0,     U    V     =  (0)(l/4)  -  0    and 
UbVb  -  (0)(3/4)  =  0  . 
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Perhaps  the  more   surprising  part  of  the  theorem  is  its  converse 

portion  -  the  assertion  that  any  pair  of  feasible   solution  for  which 

all    Q.L.  =  0     and  all  U.V,   =  0     must  be  optimal.    It  is  rather  plausible 
J     J  11 

that  an  optimal  evaluation  of  the    L's    and    V's     should  have  u;.  produce 

no  item  which  involves  a    "loss",      L,    and  should  impute  no  value  to  an 

input  which  goes  partially  unused,    but  intuitively  it  is  not  easy  to  see 

why  no  more  than  this  is  required  to  guarantee  optimality  in  a  pair  of 

solutions.      Nevertheless,    this  converse  proposition  is  valid  and  its 

proof,    which  is  not  very  difficult,    (once  Theorem  I  has  been  derived) 

is  given  in  the  appendix  to  this  chapter. 

Further  duality  theorems  tell  us  more  about  the  meaning  of 

the  dual  variables     V      and    L.     and  should  convince  the  reader  of  their 

i  J 

real  economic  significance. 

First,    let  us  examine  the  input  valuation  variables,     V.     . 
As  an  example,    suppose  input    i    is  machine  time.      Then,    as  we  have 
said,      V.     is   simply  the  dollar  value  which  is  assignee  to  an  hour  of 
time  of  this  piece  of  equipment.      Now  it  is  natural  for  the  economist 
to  feel  that  the  value  which  should  be  assigned  to  time  on  such  an  item 
is  its  marginal  revenue  product  (its  marginal  contribution  to  the  pro- 
fits of  the  firm).     And  this  is  usually  what    V.     turns  out  to  be.    As  ic 
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indicated  in  the  appendix  to  this  chapter,      V       is  equal  to  the  marginal 
profit  contribution  of  input    1  !        That  is,      V.     tells  us  (approximately) 
what  would  be  added  to  the  firm's  profits  if  somehow  the  company 
could  increase  its  available  machine  time  by  one  hour  (per  day)       More 
precisely  (wherever  the  profit  function,      II  ,    has  a  derivative)  we  have 

V    =     dll/dC 
i  i 

where     C.     is  the  total  capacity  of  input    i    (the  total  amount  of  machine 
time  available  to  the  firm).     In  other  words,    if  we  were  to  make  two 

independent   programming  calculations   -  one  calculation  with  the  aid 

3/ 
of  the  primal  problem  alone,    '      to  determine  the  value  of    3II/3C.     , 

and  another  calculation  with  the  aid  of  the  dual  to  find  the  optimal  value 

of    V.   ,     comparison  of  the  two  figures  after  both  computations  were 

completed  would  show  them  to  be  precisely  equal! 


3/    To  determine  the  marginal  profitability  of  input    i  ,    we  may  proceed 

as  follows.     Suppose  the  available  amount  of  input    i    is,    say, 

C.  =  9;  solve  the  primal  problem  for  the  optimal  value  of    II    using 

this     C      figure.     Now  substitute  for     C.    -  9    the  value     C.  +  AC.  = 

i  l  11 

9  +  ACT.     (where  some  very  small  number  may  be  used  for  the 

AC.).      Then  solve  the  primal  problem  again,    only  with  this  increase 

in  the  capacity  of  input    i  ,     to  yield  the  increased  profit  figure 

II  +  All.     It  is  now  a  straightforward  matter  to  find    All/  AC.   .     As 

i 
an  example,    the  reader  may  verify  that,    if  in  our  illustrative  pro- 
gramming problem  (31),    in  Chapter  5,    the  first  capacity  figure 
C       is  increased  from  8000  to  8001,    the  optimal  solution  matrix 
which  was  previously  given  by  (12),    in  Chapter  5,    now  becomes 
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3/  continued 


U. 


U 


8750^- 

4 

3 

4 

1 
4 

375o4~ 
4 

1 

4 

3 

4 

499^ 

1 

2 

1 

2 

n 


Q. 


Q, 


Thus    C       has  increased  by    AC     =  8001  -  8000  =  1  ,     while     II 

has  risen  by    An  =  8750  l/4  -  8^750  -  l/4  .     Hence,    we  have 

All/ A  C     =  l/4  .      Comparison  with  the  dual  value  as  given  later 

in  this  chapter,    shows  that    V     =  l/4  =      All/    AC      ,     as  our 

a  a 

theorem  asserts.     However,    it  should  be  pointed  out  that  this 

is  not  always   sure  to  happen,    for  our  theorem  tells  us  that 

V.  =  dll/dC.   ,     and  not  that    V.   =     All/ AC.     ,     i.e.,    the  theorem 

only  holds  in  the  limit,    as     AC.     approaches  zero,    and  then  only 

if  there  is  no  discontinuity  in  the  profit  function  which  would 

prevent  differentiation. 


Let  us  turn  now  to  our  other  pecuniary  variable,      L-.   ,     the 

J 

dual  slack  variable.      This  is  the  accounting  loss  per  unit  of  output    j 
That  is,    we  have  from  our  dual  constraints 


L.  =  V..  a       +  V7a       + 
J         1     lj  z    2J 


+  V     a      .   -  P. 


where    P.    is  the  actual  profit  per  unit  of  output    j     and  the  remaining 
terms  represent  the  accounting  costs  of  the  scarce  inputs  used  up  in 
producing  a  unit  of  item    j   .      By  construction,    these  accounting  costs 
are  designed  to  impute  profits  completely  to  the  firm's  scarce  re- 
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sources,    so,    at  best,    for  any  output,     j   ,     the  unit  profit    P     ,will 

J 

just  cover  the  accounting  cost,     2  V  a  .  ,       The  accounting  loss, 

i    ij 

L.   ,    associated  with  such  items,    will  be  zero.      On  all  other  items 
J 

we  will  have    L.  >  0  ,     i.  e.  ,    there  will  be  a  net  accounting  loss. 

What  is  the  meaning  of  such  an  accounting  loss?     It  only  im- 
plies that  the  inputs  used  in  producing  the  item  in  question  would  be 
more  valuable  elsewhere.      That  is,    the  profit  on  this  item  does  not 
cover  the  marginal  contribution  to  profitability  which  could  be  obtained 
by  using  the  same  inputs  in  the  production  of  some  other  goods       Thus, 
the  accounting  losses  are  only  losses  relative  to  the  most  profitable 
alternatives.      To  illustrate  the  point,    suppose  it  took  the  same  scarce 
resources  to  produce  a  handbag  and  a  pair  of  shoes  and  that  their  unit 

profitability  were,    respectively,      P,    =  $5     and    P     =  $7   .     If  these 

h  s 

were  the  only  outputs  being  considered  by  the  firm  we  would  then  have 

L     -  $2     and    L       =     0  .      That  is,    even  though   the  firm  nets  5  dollars 
h  s  s 

on  every  handbag  it  produces,     it  still  loses  2  dollars  compared  to 
the  amount  it  could  be  making  by  transferring  these  resources  to  shoe 
production.      The  accounting  loss  on  shoe  output,     L     ,     is  zero  because 
shoe  manufacturing  is  the  company's  most  profitable  alternative, 
i.  e.  ,      it  loses  no  opportunity  for  further  gain  by  keeping  bottleneck 
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inputs  tied  up  in  shoe  production.      Thus     L       and    L       turn  out.  to  be 

h  s 

what  the  economist;  has  always  called  the  opportunity  costs  of  these 
items. 

It  is  noteworthy,    then,    how  these  time  honored  concepts  of 
economic  theory,     the  marginal  product  and  the  opportunity  cost  have 
sneaked  back  into  the  analysis,      No  one  has  put  them  into  the  analysis 
of  the  primal  production  problem  which  proceeds  largely  in  terms  of 
the  relevant  physical  and  technological  considerations.     Yet,    always 
hiding  behind  this  primal  problem  is  the  dual,    which  informs  us  that, 
if  we  want  to  determine  the  optimal  magnitudes  of  our  outputs  and  the 
optimal  allocation  of  our  bottleneck  inputs,    we  should  value  the  inputs 
in  terms  of  their  marginal  (profit)  yields  and  produce  only  the  items 
for  which  the  opportunity  costs,    so  evaluated,    are  zero.      This  indic- 
ates,   in  fact,    that  no  matter  how  technocratic  the  bias  of  the  planner 
and  how  abhorrent  to  him  are  the  unplanned  workings  of  the  free 
market,    every  optimal  planning  decision  which  he  makes  must  have 
implicit  in  it  the  rationale  of  the  pricing  mechanism  and  the  allocation 
of  resources  produced  by  the  profit  system.     It  is  noteworthy  that 
these  results  have  led  to  the  open  and  well-publicized  reintroduction 
of  marginal  analysis  into  Soviet  economics  by  Russian  mathematicians 
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working  on  the  application  of  linear  programming  to  economic 
planning. 

Duality  and  Decentralized  Decision  Making. 

There  is  another  related  aspect  of  the  matter  which    has 
attracted  considerable  attention.      The  dual  accounting  prices,      V.   , 
can  serve  as  a  device  for  steering  decentralized  decision  making  along 
an  optimal  course.     Consider,    for  example,    a  firm  with  a  large  number 
of  plants  each  of  which  makes  some  use  of  overall  company  resources. 
If  plant    A     uses  more  of  the  company's  central  warehouse   space, 
less  will  be  available  to  plant    B    and  vice -versa.      Top  management 
can,    of  course,    allocate  its  bottleneck  resources  among  the  various 
plants  by  making  a  master  plan  and  deciding,    plant  by  plant,    input  by 
input,    how  much  should  go  where.      This  is  the  method  of  direct 
centralized  planning  and  decision  making. 

However,    duality  theory  points  to  an  alternative  approach. 
Suppose  management  is  in  a  position  to  calculate  the  values  of  the 
dual  accounting  prices,      V.   ,     by  an  ordinary  programming  comput- 
ation.     (The  enormous  practical  problems  likely  to  be  involved  in 
obtaining  the  required  statistics  and  the  difficulties  caused  by  non- 
linearities  must,    however,    not  be  forgotten  by  the  reader  to  whom 
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the  procedure  sounds  delightfully  simple.)     Top  management  is  now 
in  a  position  to  say  to  all  plant  directors  "you  can  have  as  much  of 
every  input  as  you  want       However,    for  every  unit  of  input  which  you 
employ,    the  company  will  debit  you  with  the  dual  value  of  the  item" 
(presumably  some  more  palatable  term  would  be  used).     It  is  to  be 
noted  that  our  second  duality  theorem  tells  us  that  items  which  should 
be  produced  in  an  optimal  solution  (i.  e.  ,    for  which  the  optimal    Q.  >  0), 
will  ordinarily  be  those  which  will  yield  no  accounting  loss  (L.  =  0). 
That  is,    only  these  items  will  yield  unit  profits  sufficient  to  cover  the 
dual  value  charge  on  the  inputs  used  to  produce  them.      Thus,    if  they 
are  effective  profit  makers,    plant  managers  can  be  left  to  decide  by 
themselves  which  items   should  be  included  in  the  product  line,  for 

commodities  which  are  not  optimal  from  the  point  of  view  of  the  com- 

4/ 

pany  will  cause  the  plant  to  incur  a  loss.  The  perfect  plant  manager 


4/     There  still  remains  the  problem  of  determining  how  much    of  each 

profitable  item  to  produce.      Under  linear  programming  assumptions 
which  rule  out  diminishing  returns  to  scale,    all  items  which  are 
profitable  should  be  produced  in  amounts  as  large  as  resources 
will  permit.      Unfortunately  this  still  leaves  a  substantial  calculation 
problem  because  increased  production  of  one  commodity  will  mean 
that  fewer  resources  will  remain  available  for  use  in  the  production 
of  other  items.      Thus,    the  dual  pricing  approach  only  takes  care 
of  part  of  the  job  of  providing  efficient  decentralized  decision  making 
arrangements.      There  are,    however,    somewhat  related  linear 
programming  methods  also  involving  dual  prices,    which,    at  least 
in  principle,    can  handle  the  entire  matter. 
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will  just  break  even  because  the  dual  accounting  prices  are  calculated 
so  as  to  eat  up  all  profits;    but  they  will  cause  no  loss  only  if  all  his 
product  line  decisions  are  optimal.      Thus  dual  pricing  can,    at  least 
in  principle,    serve  as  a  substitute  for  centralized  control  and  it  can 
open  the  way  to  efficient  decentralized  decision  making.     In  fact,    such 
an  approach  is  now  being  employed  in  some  industrial  applications 
and,    as  will  be  indicated  in  Chapter  13,    it  has  even  been  suggested  as 
an  appropriate  device  for  the  government  to  use  in  a  socialistic  econ- 
omy,   where,    it  has  been  implied,    the  method  can  help  to  attain  the 
goals  of  socialism  at  a  relatively  low  cost  in  central  control  and  loss 
of  individual  initiative. 

Solution  of  the  Primal  and  Dual  Programs. 

It  will  be  shown  in  this  section  that  our  method  for  solving 
the  primal  linear  programming  problem  immediately  yields  the   solu- 
tion to  the  dual  problem  without  further  calculation.      To  see  how  it 
does  so  we  first  write  out  a  pair  of  small  problems  in  our  standard 
form  as  follows: 
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Primal 


Dual 


max     n  =  0  +  P]Q1  +  P2Q     +  P3Q3 
subject  to 


Min    a  =  0  +  C  V     +  C?V2 
subject  to 


Ul  =  Cl  "  ailQrai2Q2   -ai3Q3 
U2  *  C2"  a21Ql  "a22Q2  "a23Q3 


Ll  "   "Pl  +  allVl  +  a21V2 
L2  "   "P2  +  al2Vl  +  a22V2 
L3  =   "P3  +  al3Vl  +  a23V2 


Notice    the  differences  in  sign  in  the  right  hand  sides  of  the  constraint 
equations  of  the  primal  and  dual  problems       This  difference  occurs 
because  in  our  original  constraint  equations  (3)  the  primal  slack 
variables,      U.   ,    appear  with  a  plus  sign  while  the  dual  slack  variables, 
L.  ,     appear  with  a  minus.     Now  we  may  write  the  two  problems  in 
matrix  form  to  obtain 


Primal  Matrix 


Dual  Matrix 


Qi 

Q2 

Q3 

0       P! 

P2 

P3 

1 

Cl    "all 

"ai2 

"ai3 

2 

C       -a 
2         21 

"a22 

"a23 

a 


L, 


Vl 

V2 

0 

Cl 

C2 

-pl 

ail 

a21 

"P2 

ai2 

a22 

"P3 

a13 

a23 
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These  matrices  exhibit  fully  the  remarkable  symmetry 
of  the  primal  and  dual  problems.     Neglecting  differences  in  signs, 
the  dual  matrix  is  merely  the  primal  matrix  flipped  over  so  that 
the  lower  left  and  upper  right  hand  corners  have  exchanged  places. 
In  addition,    there  is  a  change  in  the  sign  of  every  element  except 
those  in  the  first  column  of  the  primal  (first  row  of  the  dual)  matrix. 

These  observations  suggest  a  measure  which  can  achieve 
considerable  economy  of  notation.     All  essential  information  can  be 
preserved  in  the  following  combined  matrix: 


(5) 


n 
u. 


Q 


1 


Q. 


Q. 


0 

Pl 

P2 

P3 

cl 

"ail 

"ai2 

"ai3 

C2 

"a21 

"a22 

"a23 

a 


L. 


Several  essential  characteristics  of  this  combined  matrix 
should  be  observed. 

1)     The  combined  matrix  (5)  is  simply  the  primal  matrix 
in  (4)  with  the  dual  variables  entered  along  the  right  and  lower  sides 
to  indicate  their  position  in  the  dual  matrix  in  (4).      For  example, 


111-31 

since     L      appears  beside  the   second  row   in  the  dual  matrix  in  (4) 
it  is  written  next  to  the  second  column  in  (5).      From  (5)  we  can  at 
once  reconstruct  our  dual  matrix  by  just  interchanging  the  rows  for 
the  columns  and  reversing  the  signs  of  all  elements  except  those  in 
the  first  column  of  (5). 

2)  Each  primal  ordinary  variable  is  paired  with  the  corres- 
ponding dual  slack  variable,    and  each  primal  slack  variable  is  paired 
with  the  corresponding  dual  ordinary  variable.       For  example,    the 
third  column  corresponds  to  variables    Q       and    L       while  the  second 

row  corresponds  to  the  two  variables     U      and    V     .      We  see  that  the 

1  1 

primal  and  dual  variables  are  always  paired  in  exactly  the  same 
manner  as  they  are  in  duality  Theorem  II. 

3)  As  usual,    the  primal  variables  appearing  at  the  top  of 
the  matrix  (5)  take  the  value  zero,    in  the  current  basic  solution 

(i.  e.  ,    Q    -  Q     =  Q     =  0  ),     while  the  primal  variables  at  the  left  obtain 
their  values  from  the  first  column  of  (5)  (i.e.,    II  -  0,    U    =  C  ,  U     =  C    ) 

J.  J.  C*  C* 

In  the  original  dual  matrix  in  (4)  the  same  rule  holds,    but  because 
(5)  is  a  flipped  over  representation  of  (4)  things  appear  a  bit  inverted 
when  we  get  to  the  values  of  the  dual  variables  in  (5).     Here  the  zero 
valued  dual  variables  are  those  at  the  right   (V    =  V     =  0)    and  the 
values  of  the  dual  variables  at  the  bottom  are  given  by  the  elements 
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of  the  first  row  with  their  signs  reversed    (i.e.       a  =  0     L,    =   -P 
& '       1  r 

LZ  =   -P2>    L3  =   -P3>' 

These  three  characteristics  of  the  combined  matrix 

continue  applicable  throughout  the  simplex  calculation.      Now  consider 

some  pivot  operation,  say  the  one  represented  in  matrix  (5)  of  Chapter 

5,    which  involves  an  interchange  of  the  primal  variables     Q      and     U 

to  '12 

Suppose  in  the  dual  problem  we  were  to  pivot  in  a  way  which  inter- 
changes the  corresponding  dual  variables     L      and    V      .      It  is  easy  to 

J.  C* 

prove  by  applying  the  pivoting  rules  of  section  8  of  Chapter  5,    to  the 
dual  matrix,  that  the  new  primal  matrix  and  the  new  dual  matrix  again 
combine  into  a  single  matrix  exactly  analagous  with  (5)!     That  remark- 
able result  enables  us  to  go  through  the  simplex  procedure   step  by 
step  simply  by  earring  out  the  calculation  for  the  primal  problem 
alone.     As  an  example,    we  reproduce  as  combined  matrices  the  three 
primal  simplex  matrices  (5)  (7)  and  (12)  of  Chapter  5  for  our  illus- 
trative problem  (31)     in  that  chapter: 
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n 


u 


u. 


1st  Matrix 


Q, 


a 


L 


Q 


0 

2.  5 

2 

8000 

-1 

-2 

9000 

-3 

-2 

L. 


V. 


2nd  Matrix 


U, 


Q, 


n 

7500 

5 
"6 

1 
3 

u 

a 

5000 

1 
3 

4  ' 
3 

Q! 

3000 

1 
"  3 

2 
3 

a 

vb 

Lz 

V 


L. 


(6) 


n 


3i 

d  Matrix 

ub 

U 
a 

8750 

3 

4 

1 

4 

3750 

1 

4 

3 

"  4 

500 

1 
2 

1 
2 

Q- 


Each  of  these  matrices  satisfies  all  three  characteristics 

of  combined  simplex  matrices  which  have  just  been  described.     In 

particular,    the  first  matrix  yields  the  basic  dual  solution    U    =  V,  =  0   . 

a         b 

a  =  0  ,      L    =   -2.  5,    L     =   -2     (recall  that  these  are  obtained  by  changing 
the  signs  of  the  elements  in  the  first  row).        Similarly,    the  thira 

matrix  (which  provided  the  optimal  solution  to  the  primal  problem) 

3 
yields  the  dual  basic   solution    L     =  L     =  0  ,      a  =  8750  ,      V     =    — —   , 


Va=    I 


Ill -34 


Indeed,    this  last  basic  solution  is  also  the  optimal    solution 
to  the  dual  problem.      The  optimal  solution  can  always  be  found  in 
this  way,    from  the  first  row  of  the  optimal  combined  matrix       For, 
as  will  be   shown  next,    this  dual  solution  simultaneously  satisfies 
two  criteria: 

i    The  dual  feasibility  criterion  which  requires  all  entries 
in  the  first  row  of  the  combined  matrix  (except  for  the 
element  in  the  left  hand  corner)  to  be  negative;        and 
ii    The  dual  optimality  criterion    which  requires  all  elements 
in  the  first  column  of  the  combined  matrix  (except  for 
the  upper  left  hand  element)  to  be  positive. 
First  let  us  examine  the  logic  behind  the  optimality  criterion. 
We  do  so  in  several  steps: 

a)    It  will  be  recalled  that  in  a  maximization  problem  the 
optimality  criterion  requires  every  element  in  the  first 
row  of  the  matrix  (except  for  the  profit  figure)  to  be  negative 
because  then  only  products  with  negative  marginal  profits 
will  have  been  assigned  zero  output  levels.     Correspondingly, 
in  a  minimization  problem  (view  it  as  a  problem  of  cost 
minimization)  the  optimal  values  of  the  coefficients  of  the 
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dual  objective  function  will  all  be  positive  because  any 
process  which  has  a  negative  coefficient  (   so  that  it  reduces 
total  cost)  should  be  employed  by  the  firm  [it  should  not     be 

represented  by  one  of  the  dual  variables  at  the  top  of  our 

5/ 
original  dual  matrix  (4)] 

b)  Because  the  combined  matrix  may  be  viewed  as  the  dual 
matrix  "lying  on  its  side",    the  first  row  of  the  combined 
matrix  yields  the  basic  solutions  of  the  dual,    as  we  have 
seen,    and  the  first  column  of  the  combined  matrix  gives  the 
coefficients  of  the  dual  objective  function.      The  reader  should 
verify  that  the  initial  dual  objective  function  in  (6)  is 

a  =  0  +  8000  V  +  9000  V  corresponding  to  the  entries  0  , 
8000  and  9000  in  the  first  column  of  the  first  of  our  three 
combined  matrices. 

c)  We  conclude  that  the  optimality  criterion  for  our  dual 
problem  requires  all  elements  in  the  first  column  of  the 
matrix  to  be  positive.      This  is  because  the  first  column  of 
the  combined  matrix  corresponds  to  the  first  row  of  the  dual 

5/     For  example,    if  our  problem  is  to  minimize      a  =  90  +  3V     -  6V 
we  clearly  do  not  want    V      -  0 


Ill -36 

matrix,    and  these  are  the  elements  which  do  not  change 
sign  in  the  transition  from  the  dual  to  the  combined  matrix 
[compare  the  dual  matrix  in  (4)  with  combined  matrix  (5)]. 

The  reader  may  well  feel  at  this  point  that  something  has 

gone  seriously  wrong.      True,    the  last  of  our  three  matrices  (6) 

satisfies  the  dual  optimality  criterion,    because  the  entries  in  the  first 

column  (excluding  the     a    figure)  are  the  positive  numbers     3750    and 

500  .     But  the  other  two  matrices  in  (6)  also  satisfy  the  criterion  - 

surely  they  cannot  all  be  optimal!     The  difficulty  is  that  the  dual 

solutions  corresponding  to  these  first  two  matrices,    while  they  do 

meet  the  optimality  criterion,    must  be  rejected  because  they  are  not 

even  feasible.     We   saw,    for  example,    that  the  solution  proposed  in  the 

first  matrix  involves     L    =   -2.  5     and    L     =   -2  .      But  these  figures  clearly 

violate  the  nonnegativity  requirements      L    >    0,     L     >    0     .     Indeed, 

1  —  Z  — 

we  can  generalize  these  observations,    and  conclude  that  any  combined 
matrix  which  corresponds  to  a  non  optimal  solution  to  the  primal 
problem  must  yield  a  non  feasible  solution  to  the  dual  problem.      For 
if  the  primal  solution  is  not  optimal,    at  least   one  of  the  elements  in 
the  top  row  of  the  matrix  must  be  positive,    and  hence  (after  the  required 
change  in  sign)  the  value  of  the  corresponding  dual  variable  must  be 
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negative.      This  observation  also  yields  our  dual  feasibility  criterion, 
which  requires  that  all  entries  in  the  first  row  of  the  combined  matrix 
be  negative. 

In  sum,    when  and  only  when  we  have  found  an  optimal  matrix 
for  the  primal  problem,    the  corresponding  combined  matrix  will  yield 
an  optimal  dual  solution.      For  in  such  a  matrix  the  elements  of  the  first 
column  must  be  positive  (to  assure  that  the  primal  solution  is  feasible) 
and  the  elements  of  the  first  row  must  be    negative  (to  assure  optimahty 
of  the  primal  solution).     It  follows  that  the  dual  solution  must  be  feasible 
(because  the  elements  of  the  first  row  are  all  negative)  and  optimal 
(because  the  first  column  elements  are  positive.  ) 

Specifically,    the  solution  given  by  the  last  matrix  in  (6)  is, 
therefore,    the  optimal  solution  to  our  dual  problem,    and  it  has  been 
read  off  directly  from  the  optimal  primal  matrix  without  any  further 
calculation! 

For  the  same  reasons  it  is  possible  to  solve  the  primal 
problem  by  first  dealing  with  the  dual  problem  -  that  is,    by  solving 
the  dual  problem,    thereby  obtaining  the   solution  to  the  primal  problem 
as  a  costless  bonus.      Which  of  these  two  methods  it  pays  to  use  is  a 
matter  of  convenience  or  computational  efficiency  since  the  dual  and 
primal  problems  are  usually  not  equally  easy  to  solve. 
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The  option  of  solving  a  problem  either  directly  or  through 

its  dual  has  an  important  application  in  cases  such  as  the  primal 

problem  (2)  for  which     Q    =  Q     =       •  =Q  =0is  not  a  feasible  solution 

i  l  n 

so  that  we  cannot  use  this  as  our  initial  basic  solution       Frequently  , 

in  such  cases,    the  dual  problem  is  not  characterized  by  similar 

difficulties.      For   cuppose  in  the  primal  problem  we  have  only  "greater 

than  or  equal  to"  constraints,    such  as    0-  6M  +  2T  +  0.  7R  >    23    in 

(2)       This  requirement  is  clearly  not  satisfied  by    M  =  T  =  R  =  0 

But  in  such  a  case  the  dual  problems  will  have  only  "less  than  or 

equal  to"  constraints  such  as     9Vn  +  2V_  <    400     which  are   satisfied 

1  c  — 

by    V    =  V_    =  0  .      rlence,    in  such  circumstances,    it  is  possible  to 

solve  tne  primal  problem  by  working  only  with  the  dual  problem  and 

starting  off  with  the  ba^ic  solution  corresponding  to  the  origin  of  the 

dual  problem's  feasible  region,    where  all  the  ordinary  variables, 

V,   j    V_,  .  .  .  ,  V       >     are  equal  to  zero.     In  such  a  situation,    we   can 
1  c  m 

avoid  the  additional  work  required  by  the  use  of  the  feasioility  program 


6/    See  section  11  of  Chapter  5.     However  the  approach  just  described 
may  not  work  if  the  primal  contains  both  "less  than  or   ^qual  to"  and 
"greater  than  or  equal  to"  constraints  for  then  .the  same  its  likely  to 
be  true  of  the  dual  program. 

We  can  tell  by  direct  inspection  of  the  simplex  matrix  when  it  i  . 
possible  to  proceed  by  way  of  the  dual  in  the  manner  just  described. 
Feasibility  for  the  primal  problem  requires  all  elements  in  the 
first  column  to  be  positive.     Feasibility  for  t  i--     ua]  pro  >] 


Ill- 39 

6/  -  continued 

requires  all  elements  in  the  first  row  of  the  combined  m 
to  be  negative.      I    tie  first  of  these  condition     i     met  we  can 
proceed  by  solving  the  primal  problem  directly.     If  onl}    i 
econc    condition  i.      aiicfie*.   it  i.    utrat    git    to  proceed  via 
tne  dual.      If  it  happens,    simultaneously,    that  both  conditio 
are   satisfied  the  reader   should  verify  that  we  have  iounu  a 
matrix  whose  solutions  are  feasible  and  optimal  for  bot'i 
primal  and  dual  and  there  is  then  nothing  further  to  compute. 


which  was  invented  to  deal  with  problems  in  which  the  origin  is  not 
a  feasible  solution. 


Problems 


in  (6)? 


1.      What  is  tne  dual  basic   solution  given  by  the   second  matrix 


2.      Is  this  solution  feasible  ?     Explain, 


3.      Given  the  simplex  matrix 


n 

U. 


U2 

Qi 

ui 

300 

-2 

-4 

-1 

9 

7 

L. 


"  V2      Ll         Vl 


a.  What  is  the  corresponding  solution  to  the  primal 
p  roblem  ? 

b.  What  is  the  corresponding  solution  to  the  dual 
problem? 

c.      Show  that  both  solutions  are  feasible. 
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d.  Show  that  both  solutions  are  optimal. 

e.  Show  that  duality  theorem  I  is  satisfied. 

f.  Show  that  duality  theorem  II  is  satisfied 

4.      Why  can  the  following  not  be  a  pair  of  optimal  solutions? 
Q    -  17,    Q2  =  0,    Ul=  2,    Uz  =  4,    U3  =  0;    Lj  =  0, 


Answers 


1.     a  =  7500,      V     =  L    =  0,  V^  =    T  ,     L0  =   -    - 
a          1  b        6  2  3 

3a.      n  =  300,      U2  =  Q    =  U  =  0,     Q2  =  9,     V'3  =  7   ; 

3b.      a  =  300,     Lz  =  V3  =  0,  V2  =  2,     L    =  4,     Vj  =  1 
3e.      n  =  a  =  300 


3f.     Q  L    =  (0)(4)  =  0,     Q2L2  =  (9)(0)  =  0,    U^  =  (0)(1)  =  0,    etc 


4.       Because     U  V    /    0 
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APPENDIX:      ON  THE  DERIVATION  OF 
THE  DUALITY  THEOREMS 


This  appendix  undertakes  to  indicate   some  of  the  approaches 
which  have  been  employed  in  proofs  of  the  duality  theorems       In 
general,    the  mathematical  arguments  are  intended  merely  to  be 
suggestive  rather  than  rigorous 

Before  getting  to  the  theorems  themselves  it  is  convenient 
to  establish  a  preliminary  result  from  which  most  of  our  duality 
theorems  can  readily  be  derived.      Given  any  feasible  solutions  of  the 
primal  and  dual  problems,    this  proposition  describes  a  general 
relationship  between  the  values  of  the  primal  and  dual  objective  func- 
tions,     II    and    a  ,      in  terms  of  the  other  variables  of  the  two  problems 

(1)  a  -   n  =        2  U.V.   +     2   Q.L.     . 

In  words,    this  equation  states  that  for  any  pair  of  feasible 

solutions,    the  value  of  the  dual,      a  ,      exceeds  the  value  of  the  primal 

by  the  sum  of  the  values  of  the  primal  slacks,      U.   ,     each  multiplied 

by  the  corresponding  dual  ordinary  variable,      V.   ,      plus  the  sum  of 

the  primal  ordinary  variables,     Q.   ,      each  multiplied  by  the  corres- 

J 

ponding  dual  variable. 
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To  derive  this  equation  we  must  examine  the  constraints 
of  the  primal  and  dual  problems  which  establish  the  requirement- 
of  feasibility.      In  slack  variable  form  these  constraints  may  be  written 

Primal  Dual 


(2) 


2      a    O.    +    11=0.  2     V.a..   -    L.     =    P. 

lj    J  1  1  i   ll  1  1 

J  i 


2   a      .Q.     +    U       =     C  2     V.a.      -  L       =    P 

mi     l  mm  l   in  n  n 

J  J    J  i 


Multiplying  the  first  primal  equation  through  by    V     ,     the 
second  by     V      ,      etc.  ,    and  then  multiplying  the  first  dual  equation 
by    Q  ,      etc.  ,      we  obtain 

2   Vxa    Q    +  U  Vj    =    ClV  2   V.a.^  -  L^  =  P^ 


(3) 


2   V     a      .C    +  U     V       =  C     V         2  V.a.    Q     -  L   Q    =  P   Q 
m   mj    J  mm  mm  l    in    n  nn         nn 


Now  adding  together  the  primal  constraints  in    (3)  and  remembering 
the  dual  objective  function, 


(4)  a  =     2   C.V.     , 

i    i 
i 

we  obtain 


(5)  2    2   V  a    O.     +     2   U  V.     =      2   C  V.     =     a 

.      .       i   ij    J  .11  .ii 

i     j  J  i  i 
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Similarly,    by  adding  together  the  dual  constraints  in  (3)  and  using 
the  primal  objective  function, 


(6)  n    =     2P.Q. 

we  have 


(7)  2  2Va..Q.   -     2L.Q.  =     SP.Q.  =     n 
i    J      X   1J    J         j      J    J         J      J    J 

Since  the  first  terms  in  (5)  and  (7)  are  identical  we  obtain 

the  required  relationship     II  +     2  L.Q.  =    a  -  2  U.V     ,      which  is  clearly 

J     J  i    i 

equivalent  to  our  equation  (1)  ., 

From  (1),    we  observe  at  once 

Theorem  I   for  any  feasible   solution  to  the  primal  problem 
and  any  (not  necessarily  related)   solution  of  the  dual  we  have 
for  the  values  of  the  objective  functions     a  >    II         This  follows 
from  (1)  because  feasibility  requires 

(8)  L.  >  0,    Q    >  0,     U.  >  0    and    V    >  0    for  all    i    and    j     . 

J  ~  j  ~  i  -  i  ~ 

A   simple  extension  of  the  argument  gives  us 

Theorem  2      if,    for   some  particular  pair  of  feasible   solutions, 
we  have     a     =11       those  feasible   solutions  of  the  primal  and 
dual  problems  must  also  be  optimal, 


Ill 

Incidentally  we   shall  see  presently  that  the  converse  of  theoren 

is  also  valid  -  for  any  pair  of  optimal  solutions  we  must  have     a  -  IT  . 

To  derive  theorem  2,    observe  that  if  any  other  fea    Lble 
solution  of  the  primal  problem  yield,  a  value  of  the  objei  tive  functii 
II  ,    we  must  have,    by  theorem  1  ,      II        <  a     ,    so  that,    by  hypothe.  i 

any  such  II  <  II  ,  1.  e.  ,  II  must  be  the  maximal  value  of  II  . 
The  proof  that  en  is  the  minimal  value  of  a  is  precisely  analagou: 
and  the  reader   should  go  through  it  as  an  exercise. 

Viewed  intuitively,    theorem  1  has,    in  effect,    established 
any    a    as  a  ceiling  on  possible  values  of    II   ,      and  the  logic  of  theorem 
2    is  that  a     II    which  reaches  such  a  ceiling  (any  II     -  a   )     muut  be 
maximal.     A   similar  argument  applies  to    a       with  any     II    acting 
as  a  floor  to  the  possible  values  of    en  . 

Next,    from  (1),    we  deduce 

Theorem  3      any  pair  of  feasible  solutions   will  yield     II  -  a 

if  and  only  if,    for  each    i    and  each    j     U.V.   =  0         and 

j  1    l 

L.Q.  -  0   . 

J     J 

Proof:    by  (8)  if  any    L.Q.  /    0    that  product  must  be  positive 

J     J 

and,    similarly,    if  any    U.V.  /    0    we  must  have      U.V.   >  0   .      In  either 
y  3        i    l  ii 

such  case       2  L  O     +     2    U  V.     must  be  positive  because  it  is  a  sum 
j    J  i    1 

of  nonnegative  terms  not  all  of  which  are  zero.      Thus,    if  and  only  if 
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such  a  situation  occurs  we  must  have     a  >    II 


Q.  E.  D. 


Theorem  4     if  a  pair  of  solutions  is  optimal  they  must 
satisfy    a  =  II  . 

This  is  the  converse  of  theorem  2  and  states  that  in  every 
pair  of  optimal  solutions  the  values  of  the  primal  and  dual  objective 
functions  must  coincide. 

Outline  of  a  Proof:       consider  the  following  combined  matrix  (only 
some  of  whose  elements  are  shown  explicitly): 


Q 


U 


U 


n 

Q 
Q 
U 
U 


V 

w 

X 

k 
1 

-kz 

"k3 

-k4 

r 

k5 

s 

t 

6 
k7 

] 

U  | 

k8 

a 


L 


V 


v 


w 


V 


zero  valued  primal 
variables 


L 


L 


V 


f        zero  valued 
dual 
variables 


Aside  from  the  numoer  of  variables  included,    it  is  a  perfectly  general 
combined  matrix,    and  if  each    k  >  0     it  must  represent  the  optimal 
solution  to  both  the  primal  and  dual  problems  which  will  ultimately 
be  found  by  our  simplex  procedure.       As  was  pointed  out  in  section  5, 
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a  result  of  the  method  of  construction  of  the  combined  matrix  and 

the  nature  of  the  pivoting  process,    is  that  the  primal  variable    Q 

J 

will  always  correspond  to  the   same  matrix  row  or  column  as  the  dual 

variable,      L.   ,      while  the  primal  variable     U      is  always  associated 
J  i 

in  a  similar  manner  with  the  dual  variable    V     . 

i 

Since  the  nonzero    valued  primal  variables  appear  at  the  left, 

and  the  zero    valued  dual  variables  appear  at  the  right,    if    Q.  >  0       we 

must  have    L.  =  0    and  if    U    >  0    we  must  have      V    =  0  .     SimilarWi 
J  i  i 

because  the  nonzero  valued  dual  variables  appear  at  the  bottom  of  the 

matrix  and  the  zero  valued  primal  variables  appear  at  the  top,    it 

follows  that  if    V.  >  0    then     U.   =  0     and  if    L.  >  0    then    Q.  =  0   .      Thus, 
11  J  J 

for  every  such  pair  we  must  have    Q  L.  =  0     and     U  V     =  0  .      But  bv 

J     J  i     i 

theorem  3    it  then  follows  that  we  must  have     a  =  II.      This  completes 
our  outline  of  a  derivation 


l/    This  argument  is  actually  quite  independent  of  the  assumption  that 
the   solution  is  optimal.      In  our  method  of  computation,    every 

combined  matrix  is  constructed  to  yield    a  -  II     and    Q.L.  =  0  , 

J    J 
U.V.  =  0  .      But,    as  we  saw  in  section  5,    combined 

matrices  which  correspond  to  nonoptimal  solutions  to  the  primal 
problem  yield  nonfeasible  solutions  to  the  dual.  Thus,  the  con- 
struction involved  in  our  calculations  sacrifices  feasibility  in  the 
dual  in  order  to  guarantee  that  a  -  II  holds  throughout.  That  is 
why  only  a  single  entry  in  the  matrix  is  required  to  represent  the 
two  values,      a    and    II  . 


Ill -47 


Next,    we  come  to  the  interpretation  of  the  Oidinary  dual 

variables,      V.   .      We  have  just  seen  that,    for  any  optimal  solution, 

we  must  have 

„o  o  o  o  o 

n     =    a      =    CnVn     +    C,V_     +  •  •  •    +  C     V 

11  L     l  mm 

where  the   superscript    o  s     indicate  that  optimal  values  of  the  variables 
are  being  represented.      Except  in  cases  of  degeneracy,    which  we  con- 
tinue to  assume  away,    it  is  legitimate  to  differentiate  this  last  expres- 
sion to  yield 

Theorem  5  dTl°/dC     =     V° 

l  l 

o 
This  theorem  confirms  that    V.       is  the  marginal  profit 

l 

yield  of  input    i  . 

Theorem  6.         If  there  is  a  feasible  solution  to  the  primal 
problem  and  a  feasible  solution  to  the  dual  problem  then  both 
problems  also  possess  an  optimal  solution. 

Outline  of  Proof:      There  are  only  two  ways  in  which  a  programming 
problem  can  fail  to  have  an  optimal  solution:      a.      if  it  involves  a 
contradiction  so  that  it  possesses  no  solution,    or,    b.  if  the  value  of  the 
objective  function  is  unbounded  so  that  there  is  notning  to  prevent  it 
from  "becoming  infinite.  "     [Examples  of  problems  which  have  no 
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solution:    case  a:      Max    II  =    6x  +  3y 

subject  to 

x  >  5      and      x  -  2  <  2  . 
Since    x    cannot  be  botn  greater  than    5     and  less  tnan    •.    ,      this  pro- 
blem has  no  colution. 

Example  of  case  b:    Max    II  =  6x  +  3y 

subject  to 
x  >    5     . 
Here    x    can  be  increased  without  limit  and  so.    therefore,    can    II     . 
There  is,    then,    no  finite  maximum  value  of    II   .  ]     But  if  both  the  primal 
and  dual  problems  possess  feasible  solutions  neither  of  them  can  in- 
volve a  contradiction  (condition  a,    above).      Moreover,    let     II         and 
a       be  the  values  of    II    and    a    corresponding  to  these  solutions.      Then 
by  Theorem  1,    above,    for  any  other  solutions     II  <    a       and     II       <    a  . 
Thus  it  is  impossible  for     IT    to  increase  without  limit  or  for     a    to 
decrease  without  limit,      i.  e.  ,    both     II    and    a    are  bounded  (condition 
b).      Thus  both  our  conditions  for  the  existence  of  an  optimal  solution 
are   satisfied  and  so  both  the  primal  and  dual  problems  must  have  an 
optimal  solution. 

We  come  now  to  the  Lagrange  Multiplier  Theorem. 


Ill -.9 


Theorem  7        the  value  of    E  =    2   ,  .,ill      ,■  j      j 

"   '  3     3 

subject  to  the  inequality  constraints      2  a    O    <    C      if  and 

!J     j  -       i 

only  if  the  unconstrained    Lagrangian  expression 

(9)  II       =      S   P.Q.     -    Z    \.(S    a..Q.   -  C.) 

X  .        i    i  .       i  ij    j  i 

also  attains  its  maximum,    and  where    the     i         Lagrange 

multiplier,      X .  =  V.      ,      is  the  optimal  value  of  the     i  '      dual 
11 

variable,    so  that  the  Lagrangian  expression    (9)  becomes 


(10)  II       =      2    P.Q.    -     2    2    V°  a   .Q.     +    2    C     V°      . 

X  .11  l      ij     j  .11 

i  i       j  J     J  i 

Moreover,    the  corresponding  Lagrangian  function  for  the 

dual  problem  is  identical  with  (10)  except  that  in  the     a       expression 

X 

it  is  not  the     V.    ,     but  the    Q.     which  are  assigned  their  optimal  values, 
1  J 

Q°     . 
J 

This  theorem  tells  us,    among  other  surprising  things,    that 
the  Lagrangian  method  of  the  differential  calculus  can  also  be  applied 
to  linear  programming  problems  despite  the  fact  that  the  constraints 
are  inequalities,    and  that  they  have  not  been  changed  into  equations 

by  the  insertion  of  slack  variables! 

1 
The  correspondence  between  the  Lagrangian  expression  for 

the  primal  and  dual  problems  can  readily  be  verified  by  the  reader  by 
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forming  the     a    equation  just  as  we  derived  that  for     n       in  (10)   . 

A. 

Let  us  ignore,for  the  moment,    the  requirement  that  any  variables 
take  optimal  values  in  (10).      This  gives  us  a  generalized  Lagrangian 
expression  which  we  denote  by    L(Q,  V)  ,  1.  e.  , 


(11)  L(Q,  V)  =      2  P.Q.   -    2  2  V.  a  .Q.     +    2  C  V 

J     J  i ■     ij     J  i     i 

II     -     2    2  V.a..Q      +     a  . 
1    1J     J 

Clearly,    by  (10),    L(Q,  V°)  =     H      ,      and,    similarly,      L(Q°,  V)  =     ax    . 

Now  we  shall  prove  that  any  value  of    Q    which  maximizes     IT     subject 

to  our  primal  constraints  must  also  maximize     II     =  L(Q,    V    )   . 

X. 

By  (5)  and  (7),    we  have  for  any    a    and  any     II 


a  >    2  2  V.a..Q.  >    H  ,     ana  even  for  the  optimal    V 

~  i    ij    J  -  A  i 

2  2  V°  a  .Q.  >    n 

i      ij    J  - 

But  by  Theorems  2  and  4,    for  optimal  solutions 


(13)                a°    =      2  2  V°    a..C?  =      n° 

i        ij      J 

Thus,    by  (12)  and  (13),      H-22  V.  a..Q.  <    Ii°-  2'ZV°  a.  ,Q° 

i  ij    J  -                        i      ij     J 

o  o                        o 

Hence  coinparing           L(C,  V    )     =  11-22  V.      a..Q.  +  a 

i        ij     J 

o       o  o   „        o         ^o         o 

with                   LQ    ,  V       =  H-   23  V.    a..C.    -\-  a 
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we  see  at  once  that 

L(Q°.    V°)  >    L(Q,    V°)     . 

Thus,    it  follows  that  the  values,     Q.    ,    which  maximize     IT     must  also 

1 

maximize  the  value  of  the  Lagrangian  expression    L(Q,  V    )   =     II 

A. 

which  is  what  we  were  to  prove. 

The  corresponding  result  for  the  dual  problem  is  left  as  an 
exercise  for  the  reader.      The  converse  of  the  theorem  (i.  e.  ,    any 
values  of    Q    which  maximize     L(Q,  V    )     must  also  maximize     II 
subject  to  our  constraints)  is  a  little  more  difficult  to  prove  and  the 
derivation  will  therefore  not  be  given  here. 

This  theorem  will  be  referred  to  again  in  the  discussion  of 

the  Kuhn-Tucker  theorem  in  the  next  chapter.      However,    a  geometric 

interpretation  may  first  be  indicated  briefly.      Our  theorem  states 

that  the  optimal    Q.     will  maximize     II     =  L(Q,  V    )   .       Similarly,    the 

J  *■ 

dual  formulation  of  the  theorem  states  that  the  optimal    V.     will 

1 

o  o 

minimize     a     =  L(Q    ,  V)     .     Hence,    for     V.   =  V.       we  will  have 

K  11 

L(Q,  V)     maximal  for  optimal  values  of  the    Q's     and  for     Q.  =  Q. 
it  is  minimal  for  optimal  values  of  the      V's      .      Our  situation 
is  as  that  shown  in  Figure  1      for  the  trivial  case  where  there 
is  one  primal  variable,       Q      ,      and  only  one  dual  variable,      V     . 


111-52 


L(Q,V) 


Figure  1  . 

S    is  the  point  on  the     L(Q,  V)     surface  corresponding  to  the     optimal 

o  o 

value     Q       of  the  primal  variable  and  the  optimal  value     V       of  the  dual 

i 
variable.      Moreover,     S    is  the  lowest  point  in  the  valley    DSD    which 

is  obtained  by  varying  the  value  of    V  .      Finally,     S    is  the  highest 

t 
point  on  the  hill    H  S  H      to  which    L(Q,  V)     can  be  brought  by  varying 

the  value  of    Q  ,     It  should  be  obvious  from  the  shape  of  the   surface 

why    S    is  called  a  saddle  point. 
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A.  Future  Developments  of  Operations  Research  (Special  Topic: 
Formulating  a  Linear  Program) 

B.  Decentralized  Planning 
Decomposition  Principle  Animated 


FUTURE  DEVELOPMENTS  OF  OPERATIONS  RESEARCH 
(Special  Topic:  Formulating  a  Linear  Program) 

Before  discussing  such  a  broad  topic  as  the  future  developments  of 
operations  research,    one  has  to  note  some  historical  trends.     As  a  result 
of  the  growth  of  industry  and  of  government,    there  has  been  a  corresponding 
growth  of  information  handling,    of  computation,    and  of  analysis.     The  post- 
was  period  has  been  marked  by  advances  in  the  use  of  the  scientific  method 
for  the  planning  and  control  of  operational  systems.     Not  only  has  this  effort 
been  characterized  by  an  increasing  flow  of  information  through  operational 
and  statistical  reporting  systems,   but  also  in  the  increasing  use  of  planning 
tools  such  as  balanced  programs  and  budgets.     Many  of  the  readers  may  feel 
that  these  techniques  are  not  especially  sophisticated.     However,    they  must 
be  appreciated  as  part  of  a  movement  toward  the  simulation  of  large  scale 
systems  on  paper  and  in  machines  in  order  to  better  evaluate  the  alternative 
decision  possibilities  and  to  effect  a  better  control  of  operations. 

In  fact,    out  of  this  effort  is  emerging  a  new  science,    concerned  with 
the  problems  of  rapidly  selecting  optimal  courses  of  action  from  many  al- 
ternatives.    This  development,    in  conjunction  with  the  mechanization  of  many 
simple,    human,    control  tasks,   foreshadows  for  the  future,    the  automation 
of  many  higher  level,    mental  processes.     In  the  not-too-distant  future, 
machines  will  undertake  many  complex  control  tasks. 

The  broad  group  of  studies  dealing  with  decision  theory  and  its  ap- 
plications is  popularly  called  "Operations  Research."     It  includes  investiga- 
tions which  have  led  to  the  development  and  generalization  of  new  mathematical 
disciplines,    such  as  linear  programming,    game  theory,    dynamic  programming, 
and  the  extensions  of  older  disciplines,    such  as  mathematical  statistics. 
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And  now  a  few  words  in  behalf  of  the  support  of  research  in  Operations 
Research.     It  is  interesting  to  note  that  the  rapid  progress  in  the  use  of  math- 
ematical decision  models  since  1947  can  be  attributed  largely  to  the  fortuitous 
development  of  techniques  —  such  as  the  simplex  method  for  solving  linear 
programs.     Moreover  these  developments  were  independent  and  well  in  advance 
of  the  applications.     It  is  doubtful  if  any  of  those  who  have  ultimately  benefited 
the  most,  from  these  developments  could  have  foreseen  their  importance  with 
enough  clarity  to  support  the  basic  research  required  to  bring  them  about. 
Rather,   it  has  been  the  other  way  around—  the  developed  tool  has  inspired  the 
application. 

It  is  interesting  to  also  note  that  a  survey  of  the  present  situation 
shows  that  such  industries  as  those  processing  petroleum,   food,    and  lumber 
products  would  now  be  willing  to  undertake  full  scale  mathematical  scheduling 
of  their  systems  if  research  could  produce  a  practical  method  for  solution  of 
sufficiently  large  programs.     Yet  it  is  doubtful  if  any  company  can  or  is  willing 
to  undertake  to  support  the  basic  research  necessary  to  bring  this  about. 

To  date,    research  in  the  mathematical  programming  field  has  received 
the  major  part  of  its  suppcrt  from  the  military  and,   therefore,   has  frequently 
been  oriented  toward  military  ends.     Benefits  to  the  general  economy,    though 
greater,   have  been  in  the  nature  of  by-products. 

Moreover,   the  growth  of  applications  has  been  so  rapid  that  the 
development  of  supporting  mathematical  theory  has  fallen  behind  and  is  be- 
coming a  bottleneck  to  further  progress.     If  the  trend  toward  use  of  mathe- 
matics in  planning  and  control  is  to  continue  at  its  present  rate  of  increase, 
there  is  an  immediate  need  for  a  greatly  increased  base  of  fundamental  research 
into  mathematics  bearing  on  the  science  of  decision. 


-2- 


So  far,  the  products  of  this  research  have  found  successful  application 
to  decision  problems  of  man-machine  systems  involving  production,   allocation, 
and  distribution  under  either  deterministic,    stochastic,    or  competitive 
conditions. 

From  the  viewpoint  of  mathematics,    a  shift  in  the  origin  of  its  prob- 
lems is  seen.     In  the  years  preceding   1947,   when  operations  research  first 
began  to  develop,    mathematics  had  received  powerful  stimulation  from 
problems  arising  in  the  pure  sciences,    physics  and  chemistry.     By  contrast, 
since   1947  the  emergence  of  a  mathematics  rooted  largely  in  the  applied 
sciences  of  engineering,    sociology  and  biology  has  been  seen. 

Concurrently,    far-reaching  advances  in  computer  engineering  have 
also  taken  place.     Thus,    hand  in  hand  with  great  advances  in  the  engineering 
arts  have  come  mathematical  procedures  specifically  designed  to  exploit  the 
potentialities  of  the  enormous  computational  power  these  arts  have  made 
available.     Mathematics  is  now  playing  a  major  role,    not  only  in  these  applied 
sciences  themselves,   but  also  in  application  to  the  complex  design  and  control 
problems  of  government  and  industry. 

Now  the  problem  of  effective  control  of  a  system  is  closely  related  to 
the  question  of  what  decisions  to  make  in  order  to  improve  its  over -all  opera- 
tion.    If  a  decision  is  characterized  as  a  choice  of  actions  to  be  taken,    then 
the  problem  is  not  one  of  creating  new  technology  but  rather  it  is  one  of  picking 
some  course  of  action  out  of  the  many  existing  alternatives  in  such  a  way  as 
to  maximize  some  objective  function  which  measures  the  performance  of  the 
system  as  a  whole.     Once  such  a  problem  is  formulated  in  mathematical 
terms,    it  is,    as  we  know,    referred  to  as  a  mathematical  model.     The  math- 
ematical model.     The  mathematical  problem  is  then  that  of  finding  an  analytic 
or  numerical  solution  which  satisfies  the  conditions  of  the  model  and  maximizes 
the  objective  function. 
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One  important  class  of  decision  processes,   which  is  familiar,   is 
program  planning.     Industrial  production,    the  directed  flow  of  resources  in 
an  economy,   the  exertion  of  defense  effort  —  all  are  complexes  of  interrelated 
activities  which  require  program  planning.     The  term  "programming,"  as 
it  is  used  here,    means  scheduling  the  selection,   the  timing  and  the  extent 
of  activities  to  be  performed  so  that  a  system  may  move  from  some  fixed 
status  toward  a  desired  objective.     The  author  has,    as  may  be  known, 
specialized  in  the  field  of  linear  programming.     Here  the  interrelations 
between  the  levels  of  various  activities  throughout  the  program  satisfy  certain 
simplifying  assumptions  of  proportionality,    additivity  and  non-negativity. 

Recently  there  has  been  much  work  on  nonlinear  programming  methods. 
Here  a  few  words  regarding  this  area;    In  the  calculus,   the  derivative,   that 
is  to  say  the  first  order  linear  approximation,   plays  a  key  role.     For  this 
reason  systems  of  linear  equations  and  matrix  algebra  have  been  important 
in  the  solution  of  partial  differential  equation  systems.     Now  the  first  ten  years 
of  mathematical  programming  research  has  concentrated  on  the  solution  of 
linear  inequality  systems.     Subsequent  to  that,   the  last  five  years  has  seen 
the  emergence  of  nonlinear  programming  where  the  principal  tool  is  the  local 
approximation  by  a  system  of  linear  inequalities.     Such  an  approach  can  often 
lead  to  a  local  optimum.      When  certain  conditions  of  convexity  hold,    the  opti- 
mum solution  becomes  a  global  one.     Thus  one  natural  extension  of  linear 
programming  is  to  convex  programming.     There  are  several  other  important 
areas  under  intensive  researchwhere  linear  inequalities  are  beginning  to 
play  a  key  role.     These  are:    Integer  programming,    Network  flew  theory, 
Programming  under  uncertainty.     The  next  five  years  will  see  as  intense  a 
development  in  these  areas  as  linear  programming  has  seen  in  the  last  five 
years. 
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Linear  Programming  Defined 

One  of  the  reasons  why  the  programming  tool  has  assumed  importance, 
both  in  industry  and  in  the  military  establishment,    is  that  it  is  a  method  for 
studying  the  behavior  of  systems.     In  philosophy  it  is  close  to  what  some  des- 
cribe as  the  distinguishing  feature  of  management  science  or  operations  research, 
to  wit:  "Operations  are  considered  as  an  entity.     The  subject  matter  studied 
is  not  the  equipment  used,    nor  the  morale  of  the  participants,    nor  the  physical 
properties  of  the  output,    it  is  the  combination  of  these  in  total  as  an  economic 
process. " 

To  many  the  term  "linear  programming"  refers  to  mathematical 
methods  for  solving  linear  inequality  systems.     "While  this  may  be  the  central 
mathematical  problem  it  is  not  its  definition.     Linear  programming  is  a 
technique  for  building  a  model  for  describing  the  interrelations  of  the  com- 
ponents of  a  system.     As  such  it  is  probably  the  simplest  mathematical  model 
that  can  be  constructed  of  any  value  for  broad  programming  problems  of 
industry  and  government.     Thus  the  importance  of  the  linear  programming 
model  is  that  it  has  wide  applicability. 

Suppose  that  the  system  under  study  (which  may  be  one  actually  in 
existence  or  one  which  we  wish  to  design)  is  a  complex  of  machines,    people, 
facilities,    and  supplies.     It  has  certain  over-all  reasons  for  its  existence. 
For  the  military  it  may  be  to  provide  a  striking  force  or  for  industry  it  may 
be  to  produce  certain  types  of  products. 

The  linear  programming  approach  is  to  consider  the  entire  system 
as  decomposable  into  a  number  of  elementary  functions  called  "activities;" 
each  type  of  activity  is  abstracted  to  be  a  kind  of  "black  box"  into  which  flow 
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tangible  things  such  as  supply  and  money  and  out  of  which  may  flow  the  products 
of  manufacture  or  trained  crews  for  the  military.     What  goes  on  inside  the 
"box"  is  the  concern  of  the  engineer  or  the  educator,   but  to  the  programmer 
only  the  rates  of  flow  in  and  out  are  of  interest. 

The  next  step  in  building  a  model  is  to  select  some  unit  for  measuring 
the  quantity  of  each  activity.     For  a  production  type  activity  it  is  natural  to 
measure  the  quantity  of  the  activity  by  the  amount  of  some  product  produced 
by  it.     This  quantity  is  called  the  activity  level.     To  increase  the  activity 
level  it  will  be  necessary;,    of  course,   to  increase  the  flows  into  and  out  of 
the  activity.     In  the  linear  programming  model  the  quantities  of  flow  of  various 
items  into  and  out  of  the  activity  are  always  proportional  to  the  activity  level. 
Thus  it  is  only  necessary  to  know  the  flows  for  the   unit  activity  level.     If  we 
wish  to  double  the  activity  level,  we  simply  double  all  the  corresponding  flows 
for  the  unit  activity  level. 

While  any  positive  multiple  of  an  activity  is  possible,   negative 
quantities  of  activities  are  not  possible.     The  Mad  Hatter,   you  may  recall 
in  "Alice  of  Wonderland,"  was  urging  Alice  to  have  some  more  tea,   and  Alice 
was  objecting  that  she  couldn't  see  how  she  could  take  more  when  she  hadn't 
had  any.     "You  mean,   you  don't  see  how  you  can  take  less_  tea,  "  said  the 
Hatter,   "it  is  very  easy  to  take  more  than  nothing."  Lewis  Carrol's  point, 
of  course,   is  that  the  activity  of  "taking  tea"  cannot  be  done  in  negative 
quantity. 

One  of  the  items  in  our  system  is  regarded  as  precious  in  the  sense 
that  the  total  quantity  of  it  produced  by  the  system  measures  the  payoff.     The 
contribution  of  each  activity  to  the  total  payoff  is  the  amount  of  the  precious 
item  that  flows  into  or  out  of  each  activity.     Thus  if  the  objective  is  to  max- 
imize profits,   activities  that  require  money  contribute  negatively  and  those 
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that  produce  money  contribute  positively  to  total  profits. 

Next,    it  is  required  that  the  system  of  activities  be  complete  in  the 
sense  that  a  complete  accounting  by  activity  can  be  made  of  each  item.     To  be 
precise,   for  each  item  it  is  required  that  the  total  amount  on  hand  equals  the 
amount  flowing  into  the  various  activities  minus  the  amount  flowing  out.     Thus, 
each  item,    in  our  abstract  system,    is  characterized  by  a  material  balance 
equation  —  the  various  terms  of  which  represent  the  flows  into  or  out  of  the 
various  activities. 

The  programming  problem  is  to  determine  values  for  the  levels  which 
are  positive  or  zero  such  that  flows  of  each  item  (for  these  activity  levels) 
satisfy  the  material  balance  equations  and  such  that  the  value  of  the  payoff 
is  maximum.     It  is  clear  that  what  we  have  done  is  to  reduce  the  programming 
problem  to  a  well-defined  mathematical  problem  which  is  called  the  LINEAR 
PROGRAMMING  MODEL. 

The  Linear  Programming  Model  Illustrated 

To  illustrate  these  principles  of  the  linear  programming  approach 
to  model  building,    let  us  turn  to  an  application  in  the  petroleum  industry 
where  linear  programming  methods  have  been  very  successful.     The  compli- 
cated piece  of  plumbing  of  Fig.    1  is  a  simplified  flow  diagram  of  an  oil  re- 
finery.        The  problem  facing  management  is  this.     By  turning  valves, 
setting  temperatures,   pressures,   and  starting  pumps,   crude  oil  will  be 
drawn  from  one  or  several  oil  fields  under  the  control  of  the  refinery  (shown 
on  the  left).     Like  the  old  song  about  the  music,   it  "will  go  around  and 
around"  and  come  out  as  several  streams  of  pure  oils  (shown  on  the  right). 
The  latter  can  be  marketed  at  varying  prices.     By  changing  the  controls,    the 


Refinery  example  taken  from  a  term  paper  of  R.  J.    Ullman. 
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quantities  in  various  streams  of  pure  oils  can  be  altered.     This  will  change 
the  costs  of  operating  the  equipment  and  the  revenues  from  the  sales  of  the 
final  products.     The  various  components  are  interrelated,    however,    in  such 
a  complicated  manner,   that  it  is  not  obvious  what  is  the  best  way  to  operate 
the  equipment  to  maximize  profits.     In  spite  of  these  complex  interrelations, 
when  this  system  is  decomposed  into  elementary  functions  as  the  first  step 
in  building  a  model,   it  turns  out  that  there  are  essentially  only  three  main 
kinds  of  activities  taking  place:  Distillation,   Cracking,    Blending. 

Distillation  Activity:     The  net  effect  of  the  flash  tower,   heater, 
fractionating  towers,    strippers,    etc.,   is  to  separate  the  crude  into  varying 
amounts  of  pure  oils  of  which  it  is  composed.     Crudes  drawn  from  different 
oil  fields  will  have  different  decompositions.     Hence  there  must  be  separate 
distillation  activity  developed  for  each  type  crude.     The  maximum  amount 
of  crude  that  can  be  distilled  depends  on  which,    of  the  varying  pieces  of 
equipment  it  passes  through,  will  be  the  bottleneck.     In  our  case  we  will 
suppose  it  is  the  heater  and  that  it  has  a  fixed  capacity  of  14,  000  bbl' s  per 
day  independent  of  type  crude  processed,     From  this  description  it  is  evident 
if  the  level  of  distillation  activity  is  measured  in  number  of  barrels  of 
crude  input,   then  a  unit  level  of  activity  can  be  pictured  as  in  Fig.   2a.     It 
is  seen  that  1  bbl  of  Crude  No.    1  will  use  1  bbl  of  distillation  capacity,   and 
will  cost  $1.80  (to  purchase  and  to  distill);  the  outputs  will  be  a  stream  of 
pure  oils  in  the  amount  shown.     These  outputs  are  principally  the  heavier  oils; 
fuel,    diesel  and  stove  and  smaller  amounts  of  the  lighter  types  used  to  make 
gasoline.     If  instead  of  1  bbl,    it  is  desired  to  distill  10  or  X  bbls  of  crude, 
all  input  and  output  quantities  of  Fig.   2a  would  have  to  be  multiplied  by  10 
or  X. 
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Cracking;     The  net  effect  of  the  cracking  equipment  is  to  take  one 
of  the  heavier  type  oils  and  to  cause  it  to  be  broken  down  into  lighter  type 
oils.     In  the  case  of  fuel  oil  it  will  produce  a  small  amount  of  the  lighter  types 
and  a  larger  amount  of  stove  oil  which,    if  desired,    can  in  turn  be  re-cycled 
back  into  the  cracker  and  made  into  lighter  oils.     It  is  seen  from  Fig.   2b  that 
1  unit  of  fuel  oil  requires   1  unit  of  cracking  capacity,   and  it  will  cost  $.  16 
to  produce  the  pure  oils  in  the  amounts  shown  on  the  right.     A  separate  type 
activity  must  be  set  up  for  cracking  of  fuel,   dies  el  and  stove  oils. 

Blending;    Gasoline  is  not  a  pure  oil  but  is  a  blend  of  several  of  the 
lighter  types  of  pure  oil  (see  Fig.   2c).     It  will  be  noted  the  only  output  shown 
is  the  net  revenue  from  marketing  1  bbl  of  gasoline.     The  latter  is  assumed 
to  be  the  sales  price  at  the  refinery  less  the  cost  of  the  blending  operation. 

Once  the  flows  for  these  major  activities  have  been  determined 
on  a  per-barrel  basis,   it  is  a  simple  matter  to  set  up  the  linear  programming 
model  by  means  of  which  the  managers  can  determine  the  best  manner  to 
operate  the  refinery  to  maximize  profits.     In  Fig.   3  each  column  represents 
an  activity.     The  input  and  output  quantities  per  unit  level  of  activity  are 
shown  in  the  column;  to  distinguish  outputs  from  inputs,    outputs  are  shown 
with  a  minus  sign.     For  example,   the  data  of  Fig.   2a  is  shown  in  column 
"Distillation—  Crude  1;"  the  data  of  Fig.   2b  is  shown  in  column  "Cracking  — 
Fuel  Oil;"  the  data  of  Fig.   2c  is  shown  in  column  "Product  Marketing  — 
Gasoline."     The  other  activity  columns  are  self-explanatory.     The  amounts 
available  of  various  items  to  the  system  are  shown  on  the  right. 

The  unknown  activity  levels  to  be  determined  are  denoted  by 
x, ,  x2,  . .  . ,  x2Q  .     By  multiplying  these  unknowns  by  the  corresponding  numbers 
found  in  any  row  and  summing  the  terms  across,   the  total  obtained  should 
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equal  the  availability  shown  on  the  right. 

For  example,    the  first  material  balance  equation  reads 

l.Xj  +  l.x4   a   9500      , 

which  means  the  amount  of  Crude  No.    1  available,    9500  bbls,    is  completely 
accounted  for  by  the  amount  left  in  the  ground,     x,   ,    plus  the  amount  distilled, 

x4  ' 

The  fourth  material  balance  equation,    referring  to  the  item  distil- 
lation capacity,    reads  simply 

l.x4  +  l.x5  +  1.x,   +  1.x-    a    14,000 

which  means  that  the  distillation  capacity  of  14,  000  barrels  is  completely 
accounted  for  by  the  amount  used  in  distilling  the  various  types  of  crude  plus 
any  excess  capacity  not  used. 

Finally  the  profit  equation  states  the  revenue  obtained  from  marketing 
various  products,    ( 1.  8x,  ~  +  4.  Ox,  ,  +  4.  2x,4  +  5.  5x, ,.  +  4.  Ox, ,  +  4.  lx,  -.  + 
4.2x,q  +  4.3x,q  +  3.  3x?0)  ,   less  the  cost  of  distilling  and  crude  purchases, 
( 1.  8x.   +  1.  9xc  +  2.  Ox, )  ,    less  the  cost  of  cracking,    (.  l&Xn  +  •  2  Ixq  +  .  2  lx, , )  , 
is  the  amount  of  profit.     The  problem,   of  course,   is  to  choose  the  program  of 
activity  levels  in  such  away  that  the  material  balance  equations  are  satisfied 
and  the  profits  maximized. 
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DECENTRALIZED  PLANNING* 

I  would  like  to  talk  about  a  special  topic  related  to  some  work  of 
Philip  Wolfe  and  myself.        It  evolved  out  of  efforts  to  solve  large  scale  linear 
programs.     While  initially  it  was  conceived  of  as  a  procedure  that  takes  place 
inside  an  electronic  computer,    it  has  also  future  possibilities  for  Decentralized 
Decision  Making.     This  procedure  is  referred  to  as  the  "decomposition 
principle..  " 

Many  people  familiar  with  the  planning  process  in  large  organizations 
are  conscious  of  a  cyclic  flow  of  information  between  echelons  of  the  planning 
authority.     In  broad  outline,    a  planning  staff,    cyclically 


—••(a)    Sets  new  objectives  and  ground  rules 


(b)  Asks  lower  echelons  to  submit  proposals 

(c)  Adjusts  these  proposals  for  over-all  feasibility 


The  decomposition  principle,    discussed  here,    replaces  this  not-too-precise 
procedure  by  one  that  is  rigorous  and  well  defined. 

Economists  have  long  speculated  about  the  possibilities  of  issuing  both 
budgets  and  an  internal  set  of  prices  to  echelon  managers  to  be  used  by  them 
to  account  for  the  cost  of  running  their  respective  departments.     According 
to  economic  theory,   there  exists  an  optimal  set  of  "prices"  for  mer.  and 
materials  within  an  organization  which  would  cause  the  individual  managers 
to  make  local  decisions  which  are  optimal  for  the  system  as  a  whole.     If 


* 

This  material  was  drawn  in  part  from  my  book,    Linear  Programming  and 

Extensions,    Chapter  24,    Princeton  University  Press  (forthcoming). 

Dantzig,    George  B.    and  Philip  Wolfe,    "Decomposition  Principle  for  Linear 
Programs,"  Operations  Research,   Vol.    8(1960),   pp.    101-111. 
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these  prices  could  be  easily  determined,    this  theoretical  scheme  might  be 
an  excellent  way  of  integrating  the  above  planning  process  with  the  day-to-day 

execution. 

This  approach  gives  promise  of  replacing  current  systems  of  "accounting 
prices"  by  a  system  of  prices  that  reflects  the  over-all  goals  of  the  system. 
This  system  of  "objective-prices"  would  be  a  natural  by-product  of  the 
planning  process  itself,    as  shall  be  seen. 

The  planning  process  associated  with  the  decomposition  principle  has 
the  following  cyclic  information  pattern: 


Planning  Authority 


New  echelon  proposals  in 
the  form  of  a  summary 
list  of  needs,    detailed  plan 
is  not  required 
4 


I 


New  prices  for 
evaluating  resources 


Lower  Echelon 


Optimal  mix  of  new 
and  past  lower - 
echelon  proposals 

1 

Final  allocation  of 
resources  and  the 
setting  of  lower 
echelon  goals 


In  short,    under  this  planning  process  detalied  planning  is  delegated  to  the 
lower  echelons.     The  latter  are  expected  to  make  their  proposals  only  in 
terms  of  those  quantities  of  various  resources  which  they  require  to  carry 
out  their  goals  with  "minimum  cost."     The  resources  are  fixed  in  quantity 
and  shared  in  common  among  the  lower  echelons. 

In  general,   in  a  typical  situation,,   the  planning  authority  already  has 
in  hand  a  feasible  operating  plan  and  has  set  new  prices  on  common  items. 
This  causes  new  proposals,   which  he  then  "blends"  with  previous  proposals 
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which  have  been  submitted  on  earlier  planning  cycles.     It  can  be  shown  that  a 
relatively  small  scale  linear  program  could  provide  a  precise  procedure  for 
combining  these  proposals  in  such  a  way  that  a  new  feasible  over -all  plan  will 
emerge  with  decreased  total  cost. 

At  this  point,    the  Staff  can  decide  whether  it  wants  to  settle  for  this 
over -all  plan  or  begin  a  new  planning  cycle  to  discover  whether  the  plan  is 
optimal  or,    if  not  optimal,    to  develop  a  new  plan.     It,    of  course,    could  do 
both  by  issuing  directives  to  initiate  the  best  plan  to  date,    and  at  the  same 
time  set  up  a  new  planning  cycle. 

To  discover  whether  a  plan  is  optimal,   prices  are  first  assigned  to 
those  items  (men  or  material)  that  are  shared  in  common  by  the  various 
echelons.     These  prices  are  determined  as  an  automatic  by-product  of  finding 
the  optimal  mix  of  old  and  new  proposals.     Each  lower  echelon  is  then  in- 
structed to  propose  the  cheapest  program  to  carry  out  its  assigned  goals 
based  on  these  assigned  costs  of  procuring  the  common  resources  and  on  the 
use  of  its  own  local  facilities.     In  doing  this,    each  echelon  is  to  assume  un- 
limited availability  of  the  resources  at  the  stated  prices.     If  all  the  lower 
echelons  should  discover  that  their  most  recent  proposals  are  still  optimal, 
then  it  can  be  shown  that  the  over -all  plan  is  actually  optimal  and  the  cyclic 
process  would  terminate. 

If,    on  the  other  hand,   a  lower  echelon  should  discover  a  new  way  of 
operating  that  is  less  expensive  (according  to  the  latest  prices),   the  echelon 
is  then  instructed  to  suppress  all  detail  and  to  submit  only  a  summary  list 
of  what  it  requires  in  terms  of  the  common  resources. 

Since  each  of  the  separate  echelon-plans  was  developed  without  any 
statement  of  allocation  of  resources  (only  prices  were  announced),   it  can 
be  expected  (and  this  turns  out  to  be  true)  that  the  separate  echelon  plans 


can  have  requirements  that  are  infeasible  when  added  up  "across  the  board." 
However,    as  has  been  pointed  out  earlier,   the  staff  can  judiciously  form  a 
weighted  average  of  the  new  and  old  plans  that  (a)  maintains  over -all  feasi- 
bility,   (b)  is  lower  in  cost,    (c)  yields  new  objective-prices.     Although  this 
way  of  finding  an  optimum  plan  appears  to  have  many  points  in  common  with 
current  business  procedures,    it  differs  from  them  in  one  important  respect. 
It  could,    if  implemented,    replace  an  intuitive  process  by  one  defined  by  a 
rigorous  underlying  theory.     This  theory  prescribes  a  certain  type  of  cyclic 
information  flow  between  the  planning  staff  and  the  lower  echelons  that  yields 
a  sequence  of  improved  programs  which  approach  the  optimum;  it  suppresses 
detail  at  the  planning  level  and  furnishes  a  set  of  objective -prices  for  guiding 
the  proper  use  of  common  resources  in  the  day-to-day  execution  of  the  plan. 
The  play  that  follows  illustrates  these  points. 


-4- 


DECOMPOSITION  PRINCIPLE,    ANIMATED 

Prologue 

The  coordinator,    "Staff,"  of  the  Central  Agency  must  procure  tankers 
to  assist  his  distributor,    "Sub,"  in  the  shipping  of  their  product  from  two 
plants  to  four  terminals.     Staff  hates  details  and  has  asked  Sub  to  furnish  him 
with  only  two  numbers,    the  cost    c  ,    of  the  proposed  shipping  plan,    and  the 
number  of  tankers,     t    required  to  support  it. 

Distributor,    Sub,   has  two  arrays,    a  unit  cost  matrix    [  c.  ]  ,    and  a  unit 
tanker  requirement  matrix    [  t. .  ]    (tankers  required  per  unit  shipped): 


[Cy]      = 


r  3      6      6      5-, 


L   8      1      3      6 


;    [tij]  = 


0      0      2      0  n 


LO'    2      0      0  -j 


(The  zeros  in  the    [t..  ]   array  indicate  shipments  by  pipeline  instead  of  by 
boat.  )    Since  the  general  objective  is  to  minimize  costs,    Sub  sets  out  to  solve 
the  transportation  problem,   below,   where    a.    are  known  availabilities  at  the 
plants  and    b.    are  the  known  requirements  at  the  terminals. 


Required 


^w/ 

X12 

x13/ 

X14/ 

/        3 

6 

6 

/         5 

x2l/ 

X22x 

x23, 

X24/ 

s        8 

1 

3 

/        6 

2    =  bl      7    =   b2      3    =   b3      5    =   b4 


Available 

9    =   a 

8    =   a. 


1 


Having  taken  a  ten-day  operations  research  course  where  he  learned 
how  to  solve  small  transportation  problems,    Sub  soon  discovers  the  minimum 
cost  solution  to  this  problem  to  be 


1J J  1 


r   2       0       2       5-i 


0       7       10 


In  this  case    c    =   ZZc.x. .    =   53  ,    t    =   ZSt.  .x. .    =    18  .     Hence,    the  two  numbers 
that  Sub  furnishes  Staff,    in  his  proposed  plan  are 


Pl  = 


rep 


LtxJ 


53  -, 
18 


where    P,     represents  the  vector  made  up  of  the  components  of  his  first  plan. 

Staff  discovers,   however,    that    a  sudden  shortage  of  tankers  has 
developed,    and  the  most  he  can  muster  for  Sub  is  9  tankers.     Noting  that  two 
out  of  Sub's  eight  possible  shipping  activities  have  to  use  tankers,    Staff 
requests  Sub  to  find  a  solution  that  "goes  easy  on  the  use  of  tankers".     Sub, 
being  literal,   forgets  costs  and  immediately  comes  up  with  the  following 
solution  that  minimizes  the  use  of  tankers 


ij  J2 


0      7      0      2 
2      0      3      3 


In  this  case    Sex.    =   95    =   c,  ,    TZt.  .x. .    =   0    =  t,  .     Hence,    Sub's  new 
ij   iJ  2  ij   ij  2 

proposed  plan  is 


P2    = 


rc2,        r95, 


L  t. 


o  J 


Act  I 


Staff: 


Well,   whatever  else  one  might  say  about  Sub's  new  plan,   it  certainly 
has  gone  easy  on  the  use  of  tankers;  in  fact,    none  are  used.     But 
look  what  has  happened  to  costs  —  they  have  nearly  doubled!     I  just 
can't  tell  Sub  that  he  can  have  only  9  tankers  and  let  him  find  his 
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own  least-cost  solution;  I  tried  that  the  last  time  there  was  a  tanker 
shortage,    and  costs  soared.     Somehow  I  wish  I  had  sent  Sub  to  that 
six-week  operations  research  course,    instead  of  the  ten-day  one. 
It  would  probably  have  been  a  lot  cheaper  in  the  long  run.     NOTE: 
At  this  point,    Staff  has  decided  to  call  in  his  economist  friend, 
F.M.    Dalks,  '   as  a  consultant. 

Dalks:  According  to  good  economic  theory,   what  you  should  do  is  to  tell 

Sub  that  there  may  be  an  extra  premium  charge  for  the  use  of 
tankers.     This  will  teach  Sub  to  keep  the  costs  down  and  at  the 
same  time  not  use  tankers  excessively,    because  they  are  now  part 
of  the  charges  in  the  total  bill. 

Staff  (Enthusiastic  and  not  above  a  bit  of  subterfuge  if  it  gets  results): 

Let's  do  just  that.     (F.M.    is  crestfallen;  theory  is  theory,    but 
putting  it  into  practice  is  another  matter.  ) 

Dalks:  Let's  go  a  little  more  slowly.     It  is  not  always  easy  to  calculate 

what  the  prices  should  be  on  scarce  commodities.     It  depends  on 
many  factors.     But  let  me  assure  you  it  has  been  rigorously  estab- 
lished,   beyond  any  doubt,    that  such  prices  always  exist.     (Seeing 
distrust  creep  into  Staff's  eyes,    F.M.   begins  to  toy  with  the  idea 
of  coming  down  out  of  the  "ivory  tower".)    In  this  case,   however, 
we  have  only  one  scarce  commodity,    tankers,    and  we  could  try 
various  premium  charges  for  tankers  and  see  what  happens  to  Sub's 
use  of  them.     This  way  we  could  eventually  get  Sub  to  come  up 
with  a  plan  that  offers  both  feasibility  and  least-cost. 

Staff:  Sounds  like  a  good  approach. 

Dalks:  According  to  theory,    if  tankers  are  in  surplus,    the  price  on  tankers 

should  be  zero.     Since  Sub's  latest  plan,     P~   ,    doesn't  use  any 
tankers  at  all,    this  is  certainly  the  case.     As  a  start,   we  can  tell 
Sub  the  price,     tt    =    0  ,    on  tankers.     How's  that? 

Staff:  (Sarcastically):  Terrific!     The  only  trouble  with  it  is  that  we  have 

already  tried  placing  no  value  on  tankers,    and  Sub  came  up  with 


F.M.    Dalks,    like  the  letters  of  his  name,    is  a  composite  of  leading 
mathematical  economists  of  his  time,   who  have  applied  linear  programming 
to  economic  theory. 
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some  plan  or  other  which  we  couldn't  use.  I  threw  it  away,  but 
maybe  Marge  saved  a  copy.  .  .Ah!  See  how  valuable  a  secretary 
can  be;  here  is  a  copy.  See  Sub's  old  plan:  P.  =  53  ,  18  .  It 
isn't  feasible,    so  what  good  is  it? 

Dalks:  I  have  another  idea.     I  was  reading  an  O.R.   journal  the  other  day. 

Of  course,    I  don't  usually  read  that  kind  of  journal,   but  when  I  do 
I  find  it  is  not  so  peripheral  to  econometrics  as  some  people  think. 
Anyway,   a  friend  wanted  my  opinion  about  an  article  on  a  "decom- 
position principleo"    He  thought  it  might  throw  some  light  on  how 
to  handle  price  problems  when  there  are  breakpoints  due  to  dis- 
continuity in  the  derivatives  of  the  underlying  production  functions. 
But  this  is  beside  the  point.     The  article  suggests  that  we  should 
take  a  weighted  average  of  the  two  plans  (he  moves  to  the  blackboard) 
like  this: 


P1K1    +   PZk2    ~ 


53  95 

X.  -j    +  X. 

18  0       ' 


where    X,  +  X,    =    1  »   and,   of  course,     0        X,         1  .     For  example, 
we  could  try    L    =    1/2  ,   L    =   1/2  .     Say,    this  gives 

1/2(53   +   95)    =   74 
1/2(18   +     0)    =     9 

which  is  a  lot  cheaper  than  95  and  happens  to  use  up  the  available 
tankers.     How's  that? 

Staff:  That's  splendid,    simply  splendid.8     But  how  do  we  know  that  we  can 

average  Sub's  proposals  in  this  way?     Hew  will  Sub  know  what  to 
do?     I  don't  want  to  get  into  the  details  of  Sub's  shipping  schedule, 
you  know. 

Dalks:  Oh!  Don't  wory  about  that.     Just  tell  Sub  to  average  the  detailed 

schedules  he  used  for  generating  plans    P^     and    P?  ,   and  he  has 


Staff: 


the  answer. 


That  is  a  neat  trick  —  averaging  the  old  and  new  plans  —  and  to 
think  I  was  smart  enough  not  to  throw  away  that  old  plan.     The 
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Dalks: 


boss  is  going  to  be  very  pleased  with  I  tell  him  how  we  figured 
this  one  out.     Before  doing  so,    let's  study  this  average  plan  of 
yours  a  little  to  make  sure  it's  okay.     Let's  see,     9  tankers  and 
an  over-all  transportation  cost  of  74.     Isn't  74  rather  high,    con- 
sidering that  the  least-cost  solution  might  have  been  53,    if  it  were 
not  for  the  tanker  shortage? 

Well,    according  to  the  article,    this  does  not  necessarily  give  the 
best  solution.      What  we  are  supposed  to  do  is  set  up  a  little  linear 
program  that  tells  us  how  to  "blend"  old  and  new  plans  in  the  best 
way.     Here  it  is 


53XJ    +    95X2    +   (-z)    =    0 

=   9 


XQ   +    18X.J    +     0\2 


=    1 


(\.   >   0) 


where    XQ    is  the  slack  (if  any)  in  the  use  of  tankers.     It's  fairly 
obvious,    in  this  case,    that    X.~    =   0  ,    X.    =    1/2  ,    \?    =    1/2  ,   with 
a  cost    z     =   74     so  that  we  don't  learn  anything  we  don't  already 
know.     The  important  thing  is  that  it  gives  us  an  idea  of  what  price 
we  should  tell  Sub  to  set  on  tankers.     We  do  this  by  solving  the 
dual  linear  program. 

Staff:  I'm  afraid  you  are  in  over  my  head.     What's  the  solution? 

Dalks:  Well,   the  dual  solution  is  very  interesting.     It  works  out  this  way. 

You  see,   the  basic  variables  associated  with  the  optimal  primal 
solution  are  indicated  by  the  three  dots     •  •  •    under  the    \,   ,    \->  , 
and  (-z)    columns.     It  means  we  should  choose  multipliers 
( 1  ,    -rr     ,    -s  )    for  the  three  equations  in  such  a  way  that  when  we 
multiply  and  sum,   the    \,     and    \?    terms  vanish.     It  is  easy  to  see 
that  the  following  conditions  must  hold; 

53    +    18tt     -    s     =0      (coefficient  of    X,) 
95    +      Oir     -    s     =   0      (coefficient  of    \_) 

or     s1    =   +95    and    it1    =   +42/18    =   2y. 
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Staff:  I  don't  understand  why    tt     =    2    1/3  is  positive.     As  I  understand 

linear  programming  price-conventions,    this  means  Sub  is  paid  to 
use  tankers,    instead  of  having  to  pay  premium  for  their  use. 

Dalks:  Yes,    but  not  in  this  case.     For  once,    for  this  kind  of  problem,    the 

linear  programmers  have  set  up  their  sign  conventions  sensibly, 
just  as  we  economists  do.     So,    in  this  case,     ir    means  what  it  says. 
In  other  words,   we  tell  Sub  that  the  premium  charge  on  tankers 
might  be  2  1/3  units  per  tanker.     Let's  try  this  and  see  what  happens. 

Staff:  Quite  an  interesting  game  we  are  playing  with  Sub.      What  is  this 

s1   =   +95? 

Dalks:  Oh,    that  is  to  see  whether  or  not  we  should  try  to  blend  Sub's  pro- 

posal number  three  (when  it  comes  in)  with    P,     and    P?   .     But  we 
can  talk  more  about  this  later. 

Staff:  But  I  don't  understand.     Does  that  mean  that  Sub  is  going  to  send 

us  another  "infeasible"  proposal,   and  we  will  have  to  do  some  more 
combining  of  proposals? 

Staff:  Yes,    I  am  afraid  it  does,   unless  we  are  awfully  lucky! 

NOTE:    At  this  point  Sub  is  instructed  to  set  up  a  new  unit  cost 
matrix, 


[c..]    =   2±[t..]    = 


r  3 


U 


1J 


8      1+4- 


6  +  4- 


5  -. 
6- 


In  due  course,    Sub  arrives  at  the  following  optimal  solution  to  his 
transportation  problem: 


ij  J  3 


r  2      2      0       5-j 
0      5      3      0- 


In  this  case,     ZZc.x. .    =    57,    ZZt..x. .    =    10 
plan  he  submits,    is 


Therefore,    the  third 


P3    = 


c3n 


[t3J 


r57n 
10 


-10. 


Act  II 


Staff: 


Dalks: 


I  begin  to  understand  it  now.      What  we  do  is  set  up  a  new  little  linear 
program.     Is  there  some  special  name  by  which  we  can  refer  to  it? 

It  is  called  a  "restricted  master."     But  first,    we  want  to  test  to  see 
if    P3    is  worth  considering.     Remember     s      =    95?      What  we  do  is 
"price  out"  the  new  proposal  and  compare  it  with  95,    the  "break- 
even value."    Since  the  new  plan  has  a  transportation  cost  of  57  and 
there  will  be  a  charge  of  2+1/3  units  for  each  of  the   10  tankers  used, 
we  have    57    +    23    1/3    =   80+1/3.     This  is  less  than    s1    =    95  ,    so 
we  can  now  go  ahead  with  the  restricted  master.     Here  it  is: 

53X1    +    95\2    +    57\3    +    (-z)    =    0 

=   9 


XQ   +    18\1    +      0\2   +    10X3 


Xl    + 


X2    + 


=    1 


Staff: 


Dalks: 


Staff: 


Dalks: 


Does  it  take  very  long  to  solve?     I'm  curious  to  see  what  the  answer 
is. 

Well,    in  this  case,    no!  According  to  linear  programming  theory, 
only  two  of  the  three  plans  that  Sub  submitted  are  going  to  be  used. 
Since  no  average  of  plans  No.    1  and  3  will  have  less  than  10  tankers 
this  leaves  only  one  new  possibility,    a  combination  of    P_    and    P.,   . 
The  new  solution  is  easy.     It  is  obviously    X?    =    1/10  ,    X~    =    9/10  . 
This  gives  a  cost  of    z2    =   95(1/10)    +   57(9/10)    =    60.8. 

That's  a  big  improvement!     We  have  cut  the  cost  from  z      =    95    to 

1  2 

z      =74  and  now  to   z      =   60.  8  .     How  much  more  do  you  think  we 

can  reduce  it  if  we  keep  this  up? 

Well,    not  too  much.     It  can't  go  any  lower  than  53,   because  we  know 
that  the  value  of    z    for  the  cheapest  cost  solution  is  53  when  there 
is  no  restriction  on  tankers.     But  perhaps  we  can  do  a  little  better. 
According  to  decomposition  theory,   we  can  always  get  an  estimate 
of  a  lower  bound  by  going  back  to  the  previous  basic   solution. 
Here  is  the  formula 


I 


Min  z   >   z      -    s1    +   c    +   wt,    =    74  -  95  +  57  +  234-   =    59 


1 


11- 


Staff:  Fantastic!     If  you  are  right,    very  little  saving  below    z      =   60.8    is 

possible.     I  would  like  to  see  what  the  true  minimum  is  and  see  how 
good  your  guess  really  is. 

Dalks:  Okay,   but  it  is  not  really  a  guess.     Let  us  determine  a  new  price 

on  tankers.     Since    \?    and    \^    are  new  basic  variables,   we  have 


95    +      Ott 
57    +    19w 


s2    =0 
s2    =0 


2  2 

or     s     =95    and    tt      =    38/18    =    3.8  .     It  shows  we  should  raise  the 

1  2 

ante  on  tankers  from    it     =   2+1/3   to  -rr      =    3.8. 

Staff:  I  hope  Sub  won't  become  too  unhappy  with  our  changing  our  minds 

so  often.     He  might  start  complaining  to  the  big  boss  before  this 
noble  experiment  is  finished. 
NOTE:     In  due  course,    Sub  sets  up  his  new  array 


[c]  +   3.8[t..]    = 


r3 


L  8      1  +  7.6 


6  +  7.6      5-| 
3  6- 


and  derives  a  new  optimal  solution 


L    ij  J4 


2         7         0         0 
0        0        3        5 


Here  IZc.x..    =    87    and  ZEt..x..    =    0  .     So  this,    Sub's  fourth  plan, 
becomes 


r  c 


41    . 


L   t. 


r  87i 


0  J 


Act  III 


Staff:  This  gets  better  and  better.     Obviously  we  can  substitute   1/10  of 

P.     for  the   1/10  P_     that  we  used  earlier  and  get  a  better  solution. 
So,    let's  go  ahead  with  the  restricted  master.     Here  it  is. 
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53K,  +      95\,  +  57\7  +      87K,,  +  (-z)    =   0 

=   9 


\n  +  18\.   +  0  •   \_  +  10\,  +  0  •  \A 
0  1  2  3  4 

\,   +  X     +       \,  +  X 


=    1 


Dalks: 


Staff: 


Let  us  see  what  the  new  value  of    z    is.     Not  much  improvement, 
z3  =    57(9/10)  +  87(1/10)    =   60  .     Maybe  your  lower  bound  of  59+1/3 
is  not  so  bad  after  all.     Let's  try  your  formula  again  for  the  lower 
bound.     For  this  purpose,   we  go  back  to  the  previous  basic  solution. 
It  that  not  so?     That's  funny!     This  time  don't  we  get 

Min  z   >    z2  -   s2  +  c4  +  Tr2t4  =  60. 8  -  95  +  87. 0  =    52. 8     ? 

It  appears  that  our  previous  estimate  of  59+1/3  was  just  a  very  lucky 
one.     I  guess  we  should  make  the  lower  bound  estimate  each  cycle, 
because  some  are  better  than  others. 

I  suppose  now  we  will  have  to  get  a  new  price 

57    +    10tt3    -    s3    =    0 

87    +      Ott3    -    s3    =    0 

3  3 

or     s      =87    and    ir      =    3  .     This  time  we  tell  Sub  the  price  on 

tankers  has  dropped  to  3  . 

NOTE:  Sub's  new  problem  becomes 


r  3 


[c.  +  3t..  ]    = 


8      1+6 


6  +  6      5  - 
3  6- 


Act  IV 


Staff: 


Dalks: 


which  he  quickly  solves  because  his  last  solution,     [  x. .  ]^  ,    is  st 

optimal, 

optimal. 


optimal,     Also  he  finds  that  his  previous  one,     Lx-^]3  »    *s  also 


What  does  this  mean?     Sub  has  come  up  with  the  same  solution  as 
last  time. 

It  means  that  there  is  no  improvement.     Let  us  try  our  lower  bound 
estimate  and  see  what  it  says.     If  we  use    c^  =  87  ,   t4  =  0    in  the 
formula,   then 


-13- 


Min    z  >  z3  -  s3  +  c4  +  Tr3t4  =  60-87  +  87  +  0    =60 

If  we  use    c,  =  57  ,    t3  =  10  ,    instead  of    c4  ,   t4  ,   which  is  Sub's 
other  optimum  solution,   we  also  get 

z   >   60  -  87  +  57  +  3(10)    =   60 

Since  our  lower  bound  60  is  the  same  as  our  value  of    z  ,   this  proves 
we  are  done. 

Staff:  I  must  tell  the  boss  about  our  new  decentralized  decision  process 

and  send  out  orders  to  Sub  to  form  the  weighted  average  of  his 
plans,   namely,    .9P,  +  .  1P4  • 

NOTE:    When  Sub  carries  out  his  orders  he  finds  his  optimal  plan 
to  be 


.9[Xlj]3  +  -lfXy^  = 


r2      l\      0      4 
LO      4      3         \ 


Dalks:  Technically,   this  is  not  exactly  what  we  economists  mean  when  we 

say  "decentralized  planning."    A  better  term  would  be  "central 
planning  without  full  information  at  the  center."    A  very  interesting 
experiment!     I  may  write  a  paper  on  how  for  the  first  time  prices 
were  used  to  control,    in  a  precisely  defined  way,    a  real  life 
situation. 
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V-l 

DECOMPOSITION,    PRICING  FOR  DECENTRALIZATION 

AND  EXTERNAL  ECONOMIES 


Recently  Dantzig  and  Wolfe  [  2]  and  [3],    have  developed  a 
computational  technique,    called  the  decomposition  method,    which  in 
many  cases  permits  the  solution  of  linear  programs  of  astonishing  size. 
One  particular  applied  analysis  in  which  the  authors  of  this  paper  are 
currently  involved,    uses  this  technique  to  deal  with  a  linear  programming 
problem  containing  more  than  thirty  thousand  constraints  and  several 
million  variables! 

The  decomposition  method  can  be  given  an  economic  inter- 
pretation which  is  illuminating  in  itself.      This  article  deals  with  decom- 
position primarily  as  an  instrument  of  economic  analysis  rather  than  a 
method  of  numerical  calculation.     It  indicates  how  decomposition  takes 
advantage  of  the  economies  of  decentralized  decision  making  either  by 
divisions  within  a  firm  or  by  larger  units  in  an  economy.  The  method 


l/     The  interpretation  of  decomposition  as  a  decentralized  decision  making 
process  was  suggested  by  Dantzig  in  a  single  paragraph  in  his  forth- 
coming volume  [1].     As  far  as  we  know,    nothing  else  was  done  on  the 
matter  until  Whinston' s  recent  doctoral  dissertation  [12].     Also 
extremely  relevant  here  is  the  work  of  Gale  on  decentralized  decisions. 
See  [5]  pp.    85-93. 
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includes  in  its  operations  a  coordinating  mechanism  which  prevei 
the  decentralized  decisions  from  working  at  cross  purposes.      This 
mechanism  employs  as  its  instrument  a  generalized  interpretation 
of  the  shadow  prices  of  linear  programming  duality  theory.  That 

portion  of  the  decomposition  process  which  arranges  for  decentralized 
decisions  to  work  consistently  toward  an  overall  maximum  necessarily 
offers  an  approach  to  the  problems  which  arise  out  of  the  presence  of 
external  economies  and  diseconomies.      For  decentralized  decisions 
do  not  automatically  produce  optimal  overall  results  just  because  of 
this   sort  of  phenomenon  -  the  fact  that  a  subdivision  of  the  firm  or  the 
industry  or  the  economy  may  not  take  into  account  the  benefits  conferred 
or  the  costs  imposed  by  its  activities  on  other   subdivisions.      Decom- 
position,   then,    can  only  work  by  maintaining  control  over  these  external 
economy  or  diseconomy  problems  and  we  shall  discuss  presently  how 
this  task  is  accomplished. 


2/    Of  course,    the  association  of  dual  prices  and  decentralization  is 
by  now  a  commonplace  of  programming  theory.      See,      e.  g.  , 
Koopmans[7]  pp.  93-97,    [8],    pp.    15-23  and  Dorfman,    Samuelson 
and  Solow  [4],    pp.  59-63.     However,    externalities  cannot  be  dealt 
with  in  that  analytic  framework.      Indeed,    the  standard  programm  ng 
approach  in  decentralized  decision  making  guarantees  only  that  the 
"right'tommodities  will  be  produced.      But  it  offers  no  assurance 
that  they  will  be  manufactured  in  optimal  amounts  or  that  resources 
will  be  allocated  optimally  among  the  producers  of  these  items. 
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We  must,    however,    be  careful  not  to  overstate  the  degree 
of  autonomy  of  the  divisional  decision  maker  in  the  decomposition 
process.      True,    the  calculation  process  is  sufficiently  localized  that 
central  management  does  not  have  to  know  anything  about  the  rial 

technological  arrangements  of  the  divisions.      But  in  the  final  analysis 
the  output  decisions  are  made  and  enforced  by  the  central  planner. 
Partly  this  is  done  by  means  of  the  persuasive  powers  of  profit  and 
pricing  system.      But  at  one  stage  of  the  operation  a  central  decision, 
though  based  on  divisional  plans  and  proposals,    must,    at  least  in  the 
linear  case,    be  made  and  imposed  directly  on  the  divisions,    as  we 

shall  see. 

3/ 
This  article  begins  with  a  summary  (without  proofs)        of 

the  decomposition  theorems  and  procedures.     It  then  digresses  to 

describe  the  features  of  duality  theory  employed  in  the  process,    whose 

interpretation  may  be  of  considerable  interest  in  and  of  itself.      Finally, 

an  economic  interpretation  of  the  decomposition  process  is  given,    along 

with  some  remarks  on  its  implications  for  planning  and  control  without 

detailed  central  direction  in  the  presence  of  external  economies  and 

diseconomies.      This  discussion  is  equally  applicable  to  the  optimization 

problems  of  the  large  business  firm  which  wishes  to  delegate  some 

3/     For  derivations  of  the  theorems   see  Dantzig  and  Wolfe,    [  Z]  and  [3] 
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portions  of  its  decision  making  process  to  its  divisions,    and  to  the 
work  of  the  government  agency  which  seeks  to  arrange  for  the  act] 
freely  undertaken  in  the  private  sector  of  the  economy  to  be  consistent 
with  the  common  welfare. 

1.      Decomposable  Structures :    Illustration 

Consider  a  company  with  two  manufacturing  subdivisions. 
Division  1  is  able  to  produce  either  or  both  of  two  commodities  whose 
outputs  are  represented  by    X      and    X      .      Similarly,    the  three  potential 
outputs  of  Division  2  are  denoted  by    Y   ,    Y      and    Y      .      Both  divisions 
make  use  of  one  company  input  resource  whose  supply  is  limited  to  a 
quantity    C     .      For  example,      C      might  be  the  quantity  of  money  capital 
available  to  the  company  as  a  whole.      Division  l1  s  output  is  also  limited 
by  the  capacities     (2       and    C,     of  two  of  its  own  resources   -     e.g.,    two 
specialized  machines  which  are  not  needed  by  (or  are  not  available  to) 
the  other  division.      Similarly,    Division  Z  has  a  limited  capacity,      C      . 

This  problem  can  easily  be  restated  in  linear  programming 
terms,    letting 

P        represent  total  company  profit 

P.       represent  the  profit  per  unit  of  output    j 
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and  a..  represent  the  amount  of  input     i     requi 

to  produce  a  unit  of  output    j   , 

where  all     P.     and    a.,     are  assumed  to  be  const. mi.    as  is  usual   n 

linear  programming.      Then  our  problem  has  the  formulation 

Max  P  =  P  X.   +  P_X0   +  P_  Y,  +  P    Y_   +  P    Y 
11  2     Z  3     1  4    2  5     3 

V 
subject  to 

(1)  Wa12X2  +  auY1  +  a14Y2+a15Y3<      C 


(2) 


a21Xl  +  ^2Xi  5      C2 


a31Xl  +  a32X2  5      C3 


(3)  a43Yl  +  a44Y2  +  a45Y3  S    C4 

all    X.,     Y     >    0     . 
l  i  = 


Note  the  special  structural  characteristic  of  this  program. 
Some  of  the  constraints  contain  only  the     X    variables  and  some  contain 
only  the     Y     variables.      In  more  general  terms,    this  sort  of  structure 
can  be  described  in  the  schematic  form  shown  in  Figure  1,    where  each 
of  the    A's     represents  a  multiplicity  of  constraints. 

4/     The  decision  to  have  only  a  single  company  constraint  (1)  was  taken 
for  simplicity  in  our  illustrative  calculations.     However,    it  must 
be  recognized  that  such  a  trivial  case  avoids  most  of  the  interesting 
problems  of  decentralization.     Indeed,    in  a  linear  problem  it  is 
likely  to  mean  that  all  but  one  of  the  company's  divisions  should, 
optimally,    be   shut  down  altogether! 
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The  decomposition  algorithm  is  applicable  to  any  prog: 
winch  can  be  rewritten  in  this  form.      Fortunately,    a  number  of  impor- 
tant classes  of  problem  in  addition  to  the  one  described  also  happen  to 
share  this  structure.      Moreover,    decomposition  procedures  can  also 
be  used  in  dealing  with  some  other  types  of  problem. 


2.      Outline  of  the  Decomposition  Procedure 

Our  multi- division  company  problem  can  help  us  to  understand 


the  e 


ntire  decomposition  procedure.      The  basic  idea  is  very  simple  to 
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follow  if  it  is  envisioned  as  the  following  process:    top  management 

asks  each  division  of  the  company  to  calculate  and  submit  an  optimal 

5/ 
production  plan  as  though  it  were  operating  in  isolation.  Once  the 

plans  are  submitted,    top  management  seeks  to  modify  these  plans  to 
suit  the  company's  purposes.      By  successive  reformulation  of  these 
divisional  plans  on  the  basis  of  marginal  profit  figures  which  are  re- 
computed at  each  stage  by  top  management  and  assigned  by  it  to  the 
divisions,    an  output  plan  which  is  optimal  for  the  company  as  a  whole 
is  ultimately  arrived  at.      This  is  the  solution  to  the  original  program- 
ming problem  which  has  been  found  by  subdivision  (decomposition). 

Because  the  divisions'  plans  are  designed  without  taking 
account  of  interdivisional  interactions,    at  an  intermediate  stage  of  the 
calculation  they  will  not  be  optimal  (and  perhaps  not  even  feasible)    ' 
from  the  point  of  view  of  the  company  as  a  whole.      In  particular,    if 


5/    A  technical  point  must  be  mentioned  here.      When  a  problem  such 
as  that  given  in  section  1  is  decomposed  it  can  easily  happen  that 
the  feasible  region  for  one  or  more  divisions  is  not  bounded.      This 
would  occur,    for  example,    if  the  inequality  sign  were  reversed  in 
constraint  (3),    for  then  there  would  be  no  upper  bound  to  the  out- 
puts,   Y  ,    Y      and  Y       which  division  2  would  consider  itself  able 
to  produce.      Note  that  this  is  possible  even  though  the  feasible 
region  of  the  original  problem  is  effectively  bounded  by  the  cor- 
porate constraints.     As  a  practical  matter  this  is  not  a  serious 
problem  either  economically  or  for  computation.      No  real  division 
is  likely  to  be  without  its  own  resource  limitations,    and  in  any  case 
a  minor  modification  in  the  computational  procedures  has  been 
designed  to  cope  with  this  difficulty.      See  Dantzig  and  Wolfe  [   2   ], 
p.  .    10  5.  ■ 
6/  lie.  ,     at  such  an   intermediate   stage  there  is  no  guarantee  that  the 
divisional  plans  will  satisfy  the  corporate  constraints. 
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the  production  of  some  output  by  division    k    imposes  burdens  on 
other  divisions  by  using  up  a  scarce  company  resource,    or  by  causing 
an  upward  shift  in  the  cost  functions  pertaining  to  some  other  company 
operations,      division    k's     calculation  is  likely  to  lead  it  to  overproduce 
this  item  from  the  point  of  view  of  the  company,    because  the  costs  to 
other  divisions  will  not  enter  its  accounts.      This  is  the  classical  pro- 
blem of  external  diseconomies.      Similarly,    if  one  of  division    k's 
outputs  yields  external  economies,      i.e.  ,    if  a  rise  in  its  production 
increases  the  profitability  of  other     divisions,    division    k    will  (consider 
ing  just  its  own  gains  in  its  calculations)  not  produce  enough  of  this 
product  to  maximize  the  profits  of  the  firm. 

This  then,    is  the  basic  problem  of  decentralized  decision 
making:    to  induce  divisions  to  increase  (by  just  the  right  amount)  those 
activities  which  yield  external  economies  and  to  decrease  those  which 
produce  external  diseconomies.      The  means  whereby  this  can  be  accom- 
plished is  the  addition  to  divisional  earnings  of  a  per  unit  subsidy  or 

bonus  of  appropriate  magnitude  for  every  external  economy  yielding 

7/ 
outputs    and  a  per  unit  penalty   '     on  those  products  which  involve  disecon- 


7/     That  such  bonuses  and  penalties  can  yield  an  overall  optimum  is  of 
course  not  a  new  idea.      It  goes  back  at  least  to  Marshall  and  Pigou 
([10],  pp.    164-165)  and  is  discussed  explicityly  in  terms  of  the  firm  in 
Hirshleifer  [6],    esp.    p.  99-      Decomposition  provides  a  systematic 
theoretical  basis  and  an  explicit  method  for  computing  the  bonuses 
and  the  penalties. 
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omies, 


The  decomposition  procedure  then  proceeds  as  follows: 

1.  Each  division  submits  to  the  company  a  (suboptimal) 
plan  based  on  a  unit  profit  figure  for  each  activity  which 
has  been  assigned  by  the  company. 

2.  The  company  calculates  what  burdens  (or  benefits)  each 
such  plan  creates  for  all  other  divisions. 

3.  The  company  then  tells  the  division  manager  that  his  unit 
return  on  output    j     is  no  longer  the  original  unit  profit, 

P.   .      Instead,    he  is  given  a  revised  net    profit  figure  com- 
posed of    P.     plus  or  minus  a  per  unit  bonus  or  penalty 
equal  to  his  division's  marginal  net  contribution  to  or 
burden  upon  the  profitability  of  other  divisions. 

4.  The  divisions  submit  new  plans  which  are  again  designed 
to  serve  their  own  rather  than  total  company  interests, 
but  recalculated  in  terms  of  the  revised  net  profit  figures. 

5.  The  entire  process  is  repeated  (with  recalculated  net 
profit  figures  at  each  stage)  as  many  times  as  necessary 
for  an  optimal  set  of  decisions  to  be  arrived  at.      Since 
there  is  only  a  finite  number  of  options  (divisional  basic 
feasible  solutions)  to  be  tried,    and  since  each  step  in  the 
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procedure  is  designed  to  increase  the  profits  of  the 
company,    this  calculation  is  guaranteed  to  arrive  at  an 
optimal  solution  after  a  finite  number  of  such  steps  (see 
Dantzig  and  Wolfe,    [2]  for  proof). 
To  understand  the  process  (and,    incidentally,    to  know  when 
the  computation  terminates)  it  is  necessary  to  see  how  the  external 
economy  bonuses  and  external  diseconomy  penalties  are  calculated. 
For  this  purpose  we  must  digress  to  examine  a  more  general  class  of 
the  dual  prices  of  linear  programming  than  is  usually  discussed.      The 
remarkable  results  which  will  be  described  are  well  known  to  specialists 

in  the  subject  but  they  do  not  seem  to  be  developed  systematically  any- 

8/ 
where  in  the  published  programming  literature. 

9/ 
3.      Provisional  Dual  Prices 


We  may  designate  optimal  values  of  the  ordinary  or  structural 

variables  of  the  dual  problem  by    Vn?    V_,.  .  .  ,  V         and  the  optimal 

1         Z  m  — 

values  of  the   slack  variables  by    Ln ,    L_,  .  .  .  ,  L      .       It  is  well  known 

1         Z  n 

8/    In  our  search  for  references  we  must  acknowledge  the  help  of  Harold 
Kuhn  and  A.    W.    Tucker.      The  only  case  where  the  term  "price"   seems 
even  to  have  been  used  in  connection  with  these  more  general  dual 
values  occurs  in  Zouthendijk  [13],    p.  Z5  ff. 

9/    For  a  more  detailed  discussion  of  duality  theory,    see,    e.  g.  ,    Dorfman, 
Samuelson  and  Solow  [4],    Chapters  3  and  7. 
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that  for  an  optimal  basic   solution  of  the  production  problem  the  values 

of  these  variables  may  be  interpreted  as  follows.      V      is  the  shadow 

i 

price  or  marginal  profit  value  of  input    i    and    L.     is  the  opportunity 

cost  (relative  loss)  involved  in  producing  a  unit  of  output    j   .      That 

is,      V,     tells  the  company  how  much  its  (gross)  profit  would  increase 

if  its  investment  in  input    i     were  increased  by  one  unit  so  that 

V.   =     3P/3C.     where,    once  again,      P    represents  total  profit  and     C 
11  1 

represents  the  available  amount  (capacity)  of  input    i     .      Similarly, 
L.     tells  us  how  much  company  profits  would  be  reduced  if  it  undertook 
to  produce  one  unit  of  some  commodity,     j   ,      which  is  not  in  its  optimal 
product  line. 

It  is  to  be  emphasized  that  the  usual  interpretation  of  these 
variables  is  confined  to  an  optimal  basic   solution  of  the  primal  and  dual 
problems.      But  it  transpires  that  analogous  figures,    which  we  will 
call  provisional  dual  prices,      can  be  determined  to  correspond  to  any 


10/    Throughout  this  paper  degeneracy  is  assumed  to  be  absent.      Where 
degeneracy  occurs  this  interpretation  is  apt  to  run  into  difficulties 
because  it  may  not  be  possible  to  define  a  derivative  (marginal 
profit  figure).      For  proof  and  a  more  general  theorem  see  Mills 
[9]    pp.    186-193. 

The  marginal  profitability  figures,      V.,      will  usually  be  nonnegative 
meaning  that  added  (free)  quantities  of  any  input  will  not  decrease 
profits.      However,    if  the     i         constraint  in  the  primal  problem  is 
an  equation  rather  than  an  inequality  the  non-negativity  of    V.     is 
no  longer  guaranteed. 
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basic  feasible  solution  of  the  primal  Oproduction)  problem  whether 

or  not  that  solution  is  optimal.     In  this   section  the  results  will  merely 

be  described,    with  proofs  left  to  the  appendix  of  this  paper,    where  a 

method  of  calculation  of  our  provisional  dual  prices  is  also  specified. 

Let  our  provisional  dual  prices  be  designated  by     II FT 

1  m 

A  ,  .  .  .  ,  A     .     At  the  optimal  solution  these  become  the  dual  (shadow) 
prices  and  (dual  slack)  opportunity  cost  figures  so  well  known  in  linear 
programming  analysis.      The     II.   ,      which  are  the  provisional  dual 
values  of  the  company's  scarce  inputs,    are  called  simplex  multipliers 
and  the     A.   ,      which  are  the  provisional  opportunity  costs  of  the  outputs, 
are  called  the  simplex  evaluators. 


More  specifically,    we  have  the  following  results: 

1.      II.     is  the  marginal  profitability  of  input    i  ,    so  that 

II.   =  9P/9C,   .     However,    now  this  derivative  represents 

1  '  l 

the  addition  to  profits      P  >      which  could  be  earned  by 
increasing  the  amount  of  input    i    when  the  resource  is 


used  in  the  manner  prescribed  by  the  current  basic  feasible 
solution,      i.  e.  ,    generally,    when  that  resource  is  not 
employed  optimally. 
2.      The     A.     represent  the  opportunity  loss  incurred  in  intro- 
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ducing  the  production  of  a  commodity  less  profitable 
than  those  currently  employed  in  the  basis,    or  alterna- 
tively -  if  it  is  negative,      -A       can  represent  the  net 
gain  (the  negative  loss)   obtained  by  introducing  the  pro- 
duction of  a  unit  of  output     k     if    k     is  more  profitable  than 
any  item  currently  in  the  basis.      For  any  item     r     currently 
in  the  basis,      i.  e.  ,    any  output     r     which  is  produced  in  our 

feasible   solution,    we  naturally  have     A     -  0   . 

r 

3.       Thus,    unlike  the  optimal  values  of  these  dual  variables 

which  are  all  nonnegative,    the  provisional  dual  values,    the 
A.   ,      may  be  negative,    zero  or  positive.      .     Similarly, 
there  may  be  negative      II.     that  represent  resources  which 
would  be  used  in  the  manufacture  of  unprofitable  items 
(S)  if  the  current  basic   solution  were  adopted.      For  present 
purposes  the  possibility  of  negative     II.     and     A.     is  the 
only  difference  between  the  simplex  multipliers  and  eval- 
uators  and  the  ordinary  (optimal)  dual  prices. 


11/      In  other  words,    corresponding  to  a  non  optimal  basic   solution  of 

the  primal  problem  we  will  (usually)  have  a  non  feasible   solution  of 
the  dual. 
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4:     In  particular,    we  see  that  by  introducing  items,      k   , 

which     -A     >  0     we  can  obtain  an  improved  (more  profita 
basic  feasible  solution.      Moreover,    if  and  only  if  all 
A.  >  0     (so  that  the  introduction  of  any  item  into  the  produc  '. 
line  from  outside  the  current  basis  involves  a  nonnegative 
opportunity  loss)  we  will  have  arrived  at  an  optimal  solu- 
tion.     That  is  precisely  why  an  optimal  solution  requires 

all    A.  >  0   . 
J  - 


4.      Concepts  for  the  Decomposition  Procedure 

It  is  convenient  in  the  following  discussion  to  employ  termin- 
ology borrowed  (and  adapted  for  present  purposes)  from  the  language 
of  business  administration.      The  illustrations  will  be  drawn  from  the 
small  programming  problem  which  appears  at  the  beginning  of  this 
article. 

A  constraint  such  as  (1)  in  which  the  outputs  of  several  divi- 
sions are  involved  will  be  called  a  corporate  constraint,      while  con- 
straints like  (2)  which  affect  only  the  outputs  of  a  single  division  (in 
this  case,    division  1)  will  be  called  divisional  constraints  (in  this  case, 
division  1    constraints). 
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A   solution  to  a  division    k     programming  problem  is  an  optimal 
solution  to  the  programming  subproblem  constituted  by  the  divisional 
constraints  and  some  suitably  chosen  divisional  objective  function.      For 

simplicity,    our  discussion  will  deal  exclusively  with  a  two  division 

12/ 
case     /  and  we  will  denote  these   solutions  by     X       and     Y       for  divisions 

q  r 

1  and  2,    respectively.     Here     X       and    Y       are  both  solution  vectors 

q  r 

giving  the  magnitudes  of  the  outputs  of  each  of  the  (two)  commodities 
produced  by  division  1  and  each  of  the  (three)  items  turned  out  by  divi- 
sion 2.     Since  the  divisions  can  submit  a  (finite)  number  of  such  pro- 
posed divisional  solutions,     X.,  X.,,  .  .  .  ,  X       and    Y,,  Y_,  .  .  .  ,  Y      ,     the 

1       2  q  1       2  r 

subscripts     q    and    r     are  employed  in  order  to  indicate  how  many  such 
proposed  solutions  have  already  heen  submitted  by  each  division  up  to 
current  stage  of  the  calculation. 

Corporate  resources  are  the  available  amounts  (such  as     C 
in  our  example)  of  the  inputs  involved  in  the  corporate  constraints. 

A  divisional  resource  such  as     C    ,    C     and    C       is  defined 
analogously. 

Divisional  profit  coefficients  are  the  coefficients  of  the  objec- 
tive functions  of  the  divisional  programming  problems.      Some  or  all  of 


12/    This  involves  no  real  loss  of  generality  and  the  reader  can  readily 
translate  the  discussion  to  the  multi -divisional  case. 
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these  profit  figures  are  re-computed  in  each  cycle. 

The  executive  program  is  an  apparently  artificial  programmi 
calculation  undertaken  by  company  management  to  determine  tin    appro- 
priate values  for  the  divisional  profit  coefficients  at  the  current  stage 
of  the  calculation.     As  we  shall  see,    the  executive  program  does  take 
account  of  the  corporate  constraints.      Its  solution  takes  the  form  of   a 
weighted  average  of  the  feasible  solutions  of  the  divisional  programs. 
The  computation  of  the  executive  program  consists  of  the  calculation 
of  the  weight  or  proportion  to  be  assigned  to  each  divisional  program  in 
the   solution  of  the  entire  problem.      That  is,    if  division  1  has  so  far   sub- 
mitted the  solutions     X..,  X_.,  .  .  .  ,  X      ,     the  role  of  division  1  in  the  exe- 

1       c  q 

cutive  program  solution  will  be  assigned  by  a  weighted  average  of  these 
X's     .      However,    during  most  of  the  calculation  we  are  primarily  con- 
cerned not  with  the  solution  of  the  executive  program  but  with  the 
corresponding  values  of  the  provisional  dual  prices  which  yield  the 
divisional  profit  coefficients. 

5.      The  Executive  Program 

Suppose  the  divisions  have  sent  in,    one  at  a  time,    proposed 

divisional  solutions     X. ,  .  .  .  ,  X      ;     Y.,  .  .  .  ,  Y       to  the  corporate  accounting 

1  q  1  r 

office.      Here,    if  division  1  is  involved  in  the  production  of    n      items, 
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then    X       is  the  vector  (Xn    ,  X_,    ,  .  .  .  ,X        )     where     X,         represents 
q  lq       2q  r^q'  kq 

the  output  of  division  1' s  commodity    k     which  is  recommended  in 

division  1' s     q         solution  proposal.      Y       is  to  be  interpreted  similarly 

r 

Note  that  here  the     X.       and    Y.       are  not  variables   -  they  represent 
specific  numbers  calculated  by  the  divisions  and  transmitted  by  them 

to  the  corporate  accounting  office. 

th 
When  it  receives  the     q        proposed  solution  from  division  1  and 

the     r         proposed  solution  from  division  2  the  corporate  accounting 

office  takes  these  and  the  corresponding  data  for  previously  proposed 

solutions  and  forms  the  following  weighted  average  figures  for  each 

product    X.     and    Y.   ,     using  the  (as  yet  undetermined)  weights 

1  q  1  r 

Xa     =  |i, X.,   +  u..X.^  +   •  •  •    +  u    X. 
iq  1     ll  2     i2  q     iq 

(4) 

Y.      =  vY.n  +  v_Y.,  +  •  •  •    +  v    Y. 
jr  1    jl  2     j2  r     jr 


B 


ecause  they  are  weights,    the     u.       and    v      are  required  to  satisfy 


^  +  K.2  +  .  .  •    +  fxq  =  1 

(5)  (i,  +  v_  +  •  •  •    +  v     =  1 

v    '  12  r 


all    a     >  0  ,     all    v     >  0 

rs  -  t  - 
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These  weighted  averages     X.       and    Y.       are  then  employed  by 

1C1  jr 

corporate  accounting  as  the  variables  of  the  executive  program,    whi' 
can  now  be  described.      The  executive  program  has  three  components: 
the  objective  function  of  the  original  program,    the  corporate  constraints 
from  the  original  program,    and  the  constraints  on  the  (unknown)  weights 
(5). 

Two  characteristics  of  the  executive  program  should  be  noted: 
a)     the  divisional  constraints  do  not  appear  in  the  executive  program, 
so  that  it  usually  contains  many  fewer  constraints  than  the  original 

problem  and    b)     looking  at  the  expressions  (4)  for  the  averaged  vari- 

a  a 

ables     X.       and    Y.       of  the  master  program  we  see  that  they  are  made 
lq  jr 

up  of  two  types  of  elements-  the  unknown  weights,      u      and    v     ,    and 

i  J 

the  given  constants  such  as     X.       and    Y.        which  have  been  obtained 
iq  jr 

from  the  proposed  divisional  solutions.      It  follows  that  the  only  true 
unknowns  are  the  weights     jjl       and    v      and,    as  we  shall  see  presently, 
by  collecting  terms,    the  master  program  is  readily  rewritten  as  an 
ordinary  linear  programming  problem  involving  only  these  variables. 
Moreover,    at  any  intermediate  stage  in  the  calculation,    the  object  of 
the  executive  program  may  be  described  as  the  determination  of  an 
appropriate  feasible  (and  basic)   set  of  weights     \x       and    v      associated 
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with  each  divisional  program  so  far  proposed.      These  new  weights 
are  designed  to  yield  a  company  profit  figure  greater  than  the  returns 
corresponding  to  the  preceding  executive  program  calculation. 

An  example  of  the  construction  of  an  executive  program  should 
make  these  points  clear.      For   simplicity,    let  us  deal  with  the   single 
corporate  constraint  case  (1)  with  which  this  article  started  out 
Suppose  that  the  objective  function  and  the  corporate  constraint  are 
respectively 


(6) 


P  =  P.X.  +  P0X0  +  P    Y     +  P,Y,  +  PCY0  =  X    +  2X0  +  3Y,+  Y,,  +  2Y, 
11  2     2  31  42  53  1  2  12  3 

and 

anX1+  a12X2+a13Y1+a14Y2+a15Y3  =  4X,  +  X2  +  2Y     -  5Y2  +  Yj  <  C, 


Suppose,    moreover,    that  three  solutions  have   so  far  been  proposed  by 
division  1  and  two  solutions  have  been  proposed  by  division  2  as  follows 


(1) 


< 


Division  1  proposed  solutions 
1st  proposal  2nd  proposal  3rd  proposal 


xn  =  7 

X2!  =  4 


\z-z 


X22  =  ° 


X13  =  6 


X23=5 


Division  2  proposed  solutions 
1st  proposal  2nd  proposal 


Yu=0 

Y21  "  12 
Y31  =  0 


Y12  =  ° 

Y22  =  ° 
Y32  =  9 
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By  direct  substitution  of  values  from  the  proposed  divisional  solutions 
(7)  our  weighted  average  variables  (4)  become  (omitting  their  last 
subscript  for  simplicity) 


(8) 


?M1  +  ZhL2  +  6t^3 


< 


4^ 


12v. 


+    5|JL. 


9v 


Now,    our  executive  program  is 

Maximize  P  =  P,xf  +  P.,X^  +  P^Yf  +  P    Y^  +  PCY^ 

11  L     L  31  42  o     J 


subject  to 


allX"  +  ai2X2  +  ai3Y"  +  ai4Y2  +  al5Y3    ±  Cj 


v     +  v 
12 


=  1 


all    ix     >  0  ,     all    v    >  0  . 

Substituting  in  the  values  of  the     P.     and    a.,     from  (6)  and  the  values 

°  i  ii 

Pi  PL 

of  the    X      and    Y.     from  (8)  the  first  two  relationships  become 
i  J 

P  =  (7p.  +  2(jl2+6Hl3)+2(4Hl1+5Hl3)   +  12vl+2(9v2)   =  15Hl;l+2^2  +  16Kl3  +  12vi+18vz 
4(7Hl1+2Hl2+6Hl3)  +(4Hi1+5M.3)-5(12vi)  +  9vz=  SZ^^  +  Z^-SOv^v  z<  Cy 
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13/ 
Hence,    in  final  form,    the  executive  program  is 

Maximize  P  =  15u    +  2u     +  l6u     +  12v     +  18w  ) 

subject  to 

32^  +  8(jlz  -l    29^3   -  60vi  +  9vz  <    C 

v1+     vz  =  l 
^  >  0  ,     kz  >  0,     p.3  >  0,    v    >  0,    v2  >  0     . 

This  has,    indeed,    changed  fromaproblem  in  the  variables     X      and    Y 

1  J 

into  one  for  the  determination  of  improved  feasible  weights     u.       and    v 

&  "s  t 

to  be  used  in  forming  the  average  of  the  previously  proposed  divisional 
solutions. 

It  will  be  noted  that  the  number  of  variables  in  the  executive 
program  increases  as  additional  proposed  divisional  solutions  are 
entered  into  the  records.      Whenever  a  division  submits  a  new  proposed 
solution  and  the  corporate  accounting  office  computes  its  resource  use, 
another  weight  variable  is  added  to  the  master  program.      Thus,  when 

13/    Note  that  the  coefficients  of  the  divisional  constraints  play  no  direct 
role  in  the  formulation  of  the  executive  program.      This  confirms  our 
earlier  assertion  that  decomposition  require  s  no  knowledge  on  the 
part  of  central  management  about  the  nature  of  the  internal  techno- 
logical arrangements  of  the  divisions.      In  this  respect  the  procedure 
offers  very  real  advantages  of  decentralized  decision  making. 


v-zz 


division  1  sends  program    X      ,     to  the  company  accounting  office,    then 

q+1 

14/ 
u.  will  be  added  to  the  set  oi  variables  of  the  executive  program. 

6.      Description  of  the  Decomposition  Procedure 

Assume  now  that,    up  to  this  stage,    division  1  has  submitted 
q-1    and  division  2  has  submitted    r-1    proposed  solutions  and  the     q 
and    r         proposed  solutions  have  now  been  received  by  the  corporate 
accounting  office. 

By  one  pivot  step  of  the  simplex  calculation  an  improved 
feasible  solution  for  the  executive  problem  is  then  found.      However, 
the  solution  values  of  the  variables  in  this  program  are  disregarded 

until  the  final  iteration  which  gives  the  optimal  solution  to  the  original 

15/ 
(decomposed)  programming  problem.  Instead,    the  purpose  of  this 


14/     In  practice,    as  a  matter  of  computational  convenience,    at  any  stage 
of  the  calculation  many  of  the  previously  proposed  divisional  solu- 
tions and  the  corresponding  weight  variables     u       and    v      may  be 
disregarded  in  the  executive  program.      The  computation  can  be 
arranged  so  that  at  any  stage  a  proposed  solution  which  is  given  a 
zero  weight  by  the  executive  program  calculation  is  dropped  from 
the  next  executive  program  calculation.  I.e.,    if  weight  (jl.     of  pro- 
posed divisional  solution    X.     happens  to  be  equal  to  zero  at  some 
stage,    then    X.     is  dropped  from  the  executive  program  before  the 
following  iteration. 
15/    As  described  in  the  preceding  footnote,    the   solution  of  the  executive 
program  at  any  intermediate  stage  of  the  computations  may  also  be 
used  to  determine  which  proposed  divisional  solution  can  be  dropped 
from  the  next  stage  of  the  calculation.     A  proposed  divisional  solution 

whose  weight,      u       or      v.   ,    turns  out  to  be  zero  will  be  omitted  from 

.    l  J 

the  next  executive  program. 
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computation  is  to  determine  the  provisional  dual  prices  (simplex 

multipliers),      II  =  (II  ,  ...  ,  II      ,    II,    IT    ),     for  each  of  the  corporate 

1  m         1         Z  ^ 

resources.      The  last  two  of  these  dual  prices,      II      and     II      ,    corres- 
pond,respectively,    to  the  constraints      Z)|jl    =  1      and     Sv    =  1  . 

1  J 

Using  these  provisional  dual  prices,    revised  divisional  unit 
profit  figures  are  then  calculated  for  this     q        and    r         proposed  divi- 
sional solution,    in  accord  with  the  following  expressions: 

(9)  R    .  =  P.    -(  n  a.  +n    a.    +  •  •  • +n     a      .)  and  S    .  =  P.-(II  a  .  +  11    a     +•  •  • +11    a       J 

qi  i  1   li        Z    Zi  m    mi  rj       j        1  lj       2    2j 

These  divisional  revised  profit  figures  are  to  be  employed  in  the  next 
round  of  calculations  as  the  coefficients  of  the  divisions'   objective 
functions 

(10)  R     =     2  R    .X.     and    S     =SS    .Y.     . 

q  qi     l  r  rj     J 

The  revised  profit  figures  (9)  will  presently  be   shown  to  differ  from  the 
ordinary  profit  coefficients,    the     P.,    by  the  addition  of  bonuses  and 
penalties  imposed  upon  activities  for  their  use  of  company  resources. 
Note  that  the  last  two  entries  in  the     II    vector,    the     II      and     II       , 
do  not  have  any  role  in  determining  the  revised  profit  figures. 

These  revised  divisional  profit  figures  are  sent  to  divisions 
1  and  Z  who  then  compute  the  optimal  solution  to  the  divisional  linear 
programs  which  are  composed  of  the  objective  functions  (10)  together 
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with  the  divisional  constraints. 

The   solutions  are     X    ,.     and    Y      ,  ,      i.  e.  ,    they  are  the  pro- 

q+1  r+1 

posed  divisional  programs  for  the  next  iteration.     As  before,    these 

solutions  are  then  transferred  to  corporate  accounting  which  comput< 

the  new  weighted  average  solutions  (4)  to  be  used  in  the  next  master 

program  calculations. 

The  test  for  optimality  may  now  be  described.      It  is  in  this 

test  that  the  last  two  shadow  prices,      II      and     II       ,    are  employed. 

For  this  purpose  the  total  revised  profit  contribution  figures  for  the 

two  divisions,     R       and    S       as  given  by  (10),    are  compared  with     II 
q  r  1 

and     II      ,    respectively.      If  either    R     >  IT     or    S     >  II.,   ,    or  both,    then 
Z  q  1  r  Z 

the  new  divisional  proposed  solution  improves  the  company's  profit 

position.     In  this  case  the  decomposition  procedure  introduces  one  of 

these   solutions  into  the  executive  program  and  goes  on  to  the  next 

iteration,    repeating  the  calculations  until  an  optimal  solution  is  reached. 

On  the  other  hand,    if    R     =11,     and     S     =  IK     then  an  optimal  solution 

q  1     r  Z  r 

to  the  executive  problem  has  been  found.      That  is,    the  weights     p. . 
and    v.     determined  by  the  executive  program  calculation  of  the  pre- 
vious iteration  have  now  been  shown  to  be  optimal.      The  optimal 
solution  to  our  original  programming  problem  is  given  by  the  output 

levels     X*    and    Y*      which  are  obtained  by  the  indicated  averaging  of 
i  J 
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the  proposed  divisional  solutions: 

X*   =  v*X      +  u*X      +  •  •  •    +  |j,*X.      ;    Y*=  vff..  +  v*Y.,+  •  •  •    +  v*  Y.      . 

1  1      ll  2     l2  nq     iq  j  1     jl  2      jZ  r      jr 

This  then,    in  bare  outline,    is  the  essence  of  the  decomposition  pro<  edure 

Let  us  now  try  to   see  why  it  works. 

7.     Interpretation  of  the  Executive  Program 

Let  us  next  examine  the  artificial  revised  objective  function 

which  is  assigned  to  the  division  by  central  management.     At  each  stage, 

rather  than  maximizing       2  P.X.   ,     the  first  division  is  told  to  maximize 

11 

the  revised  objective  function     2R    .X.   =  2  (P.   -   2    n  a   .)X      which  re- 

qi  i     j     j      .     i  ij    j 

places  the  initial  objective  function 

2   P.X      . 

j    J  3 

The  revised  profit  function  differs  from  the  original  one  by  the  amount 

-  2    2    n.a..X.       . 
1    iJ    J 

The  economic  meaning  of  this  expression  can  readily  be  inferred  when 
one  recalls  that  we  interpret    X.     as  the  quantity  of  output    j   ,      a.,     as 
the  amount  of  input    i    needed  to  produce  a  unit  of  output    j     and     II. 
as  the  value  or  marginal  profitability  (to  the  firm)  of  a  unit  of  (cor- 
porate) input    i  .      Thus        211  a..X.     is  the  total  value  of  all  the  inputs 

i      i    ij     J 

going  into  the  production  of  output    j     and 
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-  2    2    n.a  .X. 

■  •  !      1J       J 

1        J 

is  the  opportunity  cost  to  the  firm  of  all  resources  used  by  division  1 

in  producing  all  its  outputs.      In  sum,    at  each  iteration  the  division  is 

told  to  maximize  not  its  own  profit,    but  rather  a  net  figure,      2  R      X      , 

2 — qi  —i 

which  subtracts  from  the  profit  figure  the  value  to  the  rest  of  the  firm 
of  the  scarce  inputs  that  the  division  uses. 

Thus  the  division  is,    in  effect,    forced  to  pay  a  penalty  for  the 
external  diseconomies  which  its  outputs  impose  on  the  rest  of  the  firm. 
It  should  be  added  that  in  those  cases  where  a     II       happens  to  be  nega- 

K. 

tive  (indicating  that  at  this  iteration  an  input    k    involves  a  disutility 

to  the  rest  of  the  firm),      -II,  a,  .X.     will  be  positive,    so  that  the  division 

k   kj    j 

receives  a  bonus  for  using  up  this  nuisance  resource.      That  is,    the 
division  is  compensated  for  any  external  economies  which  it  brings 
about.      We  see,    then,    how  the  decomposition  procedure  provides  each 
division  with  net  profit  figures  which  are  designed  to  compensate  for 
the  external  economies  and  diseconomies  caused  by  its  activities.      This 
is  the   secret  of  the  procedure,    the  means  whereby  it  achieves  results 
that  are  optimal  for  the  operation  as  a  whole,    though  decision  making 
remains  decentralized,    at  least  in  form. 

Now  let  us  look  into  the  workings  of  the  optimality  test  of 
the  decomposition  method.      It  will  be  recalled  that  in  our  two  divisions 
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case  there  are     m+2     constraints  in  the  executive  program.      The  first 

m    of  these  correspond  to  the     m     scarce  company  input  resources  for 

whose  use  the  two  divisions  of  the  company  compete.      Corresponding 

to  these     m     scarce  resources  there  are  the     m    ordinary  simplex  mulu 

pliers     II  ,  .  .  .  ,  II         .      In  addition,    there  are  two  constraints      2  u    -  1 
1  m  ni 

and    2v.  =  1    which  are  associated  with  the  multipliers      II      and     II       . 
J  ^12 

The  two  last  constraints  serve  to  require  us  to  select  such 

weights  that  the     X.       and    Y.       variables  (4)  to  be  used  in  the  master 

program  are  true  averages     of  the  divisions'  proposed  solutions.      To 

see  what  this  implies,    suppose  the  constraint     2  u.   =  1    were  replaced 

by     2ii,  >  1  .      This  would  mean  that  the     X.       employed  in  the  current 
l  —  iq 

solution  of  the  executive  program    could  exceed  a  weighted  average  of 
the  previously  proposed  divisional  programs  for  division  one.      II 

may,    now,    be  interpreted  as  the  marginal  profitability  of  a  (unit) 
increase  in  the  capacity  figure  (the     C)     associated  with  the  corres- 
ponding constraint,     2  jjl.   =  1.      But  here  the     C    is  obviously  equal  to 
unity  to  begin  with.      Hence     II      must  be  interpreted  as  the  marginal 
profitability  of  letting  the     (jl.'s     add  up  to  a  total  greater  than  unity. 
As  we  have  just  seen,    this  is  tantamount  to  having  division  1    increase 
its  outputs  beyond  a  weighted  average  of  its  previously  proposed  pro- 
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grams,    as  a  result  of  which  it  must  use  up  more  of  the  company  re- 
sources.     In  other  words,      II      may  be  considered  the  relative  marginal 
profitability  of  a  transfer  of  some  company  resources  to  division  1  , 
and    II       is  the  corresponding  figure  for  division  Z.     L 

We  can  now  turn  directly  to  the  optimality  test  and  the  role  of 
II      and    II       in  that  criterion.        Whenever  the  division  completes  another 
suboptimal  calculation  based  on  its  revised  objective  function,    it  will 

have  determined  on  a  new  proposed  divisional  solution    X     =(X      ,  X 

q  ql       q2 


,  .  .  .  ,X         )   .      This  proposed  solution  must  next  be  compared  in  profit - 
qn  . 

ability  to  the  company  with  the  programs  which  the  division  had  pre- 
viously prepared.  It  will  be  remembered  that,  for  this  purpose,  the 
division's  revised  profit  figure 

(11)  R    (X   )  =     S(P.   -  2    n.a   .)X    . 

q    q       j     j     i     i  ij    qj 

which  is  yielded  by  the  proposed  program  X  ,  is  compared  with  the 
value  of  II  ,  the  simplex  multiplier  associated  with  the  master  pro- 
gram constraint      2u.  -  1  .      If    R     =11.,   (and  if  corresponding  conditions 

"l  q  1  j-  t. 

16/    More  specifically  II      is  the  profit  to  be  earned  from  a  100  per  cent 
increase  in  the  company  resources  made  available  to  division  1  . 
For,    starting  with    2  p..   =  C  =  1,    a  unit  increase  in    C     gives  us 
2  jjl .  =  2  .      Thus,    since  in  a  linear  programming  problem  returns  do 
not  diminish,      II    is,    in  effect,    the  total  profit  contributed  to  the 
company  by  division  1  under  the  terms  of  the  solution  currently  pro- 
posed by  the  executive  program.      Similar  remarks,    of  course,    apply 

to  n 
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are   satisfied  for  each  division)  the  current  (preceding)   solution  of 

the  executive  program  is  judged  to  be  optimal,    whereas  if    R     >    IT 

q         1 

a  new  improved  executive  program  can  be  obtained  by  taking  the  most 

recently  calculated  divisional  program    X       and  averaging  it  in  with 

divisional  programs  which  had  previously  been  proposed. 

But  we  have   seen  in  this  section  that  the  revised  profit  figure  , 

R   (X    )   ,    is  a  measure  of  the  net  profit  contribution  which  division 
q      q  c 

l's     proposed  new  solution    X       offers  to  the  company,    while     II      is 
q  1 

a  measure  of  the  profit  contribution  offered  by  the  division's  previously 

proposed  solutions.     Hence,    the  comparison  of    R       with     II      is  simply 

an  examination  of  the  net  profitability  of  the  new  proposed  solution    X 

in  relation  to  the  profits  which  are  offered  by  the  optimally  weighted 

average  of  the  previously  proposed  divisional  solutions.      If    R       exceeds 

II     ,     the  potential  net  gain  from  the  introduction  of  the  new  solution, 

R      ,    is  greater  than  the  marginal  yield,      IT  ,      of  the  assignment  of 
q  1 

additional  company  resources  to  division  1  to  be  used  in  accord  with 
the  averaged  previous  solutions.     In  other  words,    the  new  solution 
offers  a  higher  marginal  yield  than  the  old,    and  the  company  will  gain 
by  taking  some  resources  out  of  the  use  required  by  the  previous  sol- 
ution and  allocating  them  to  the  output  pattern  described  by  the  new 
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solution.      In  this  case  it  pays  to  compute  a  new  executive  program  which 
averages  in  (a  more  profitable  manner)  this  new  proposed  solution 
along  with  those  that  preceded  it,    and  to  proceed  with  further  iterations 
of  the  decomposition  algorithm  until  the  optimality  criterion  is  finally 
satisfied. 

One  more  observation  may  help  to  confirm  our  interpretation  . 
Consider  any  division  1  proposal,      X      ,    which  is  currently  being  aver- 
aged with  nonzero  weight     \±       into  the  combined  executive  program 

ir 
•a 

solution    X       as  given  by  (4).      In  other  words,      X       is  a  previously 

q  p 

proposed  division  1  solution  which  has  not  been  rejected  by  the  master 

program  (it  has  not  been  assigned  a  weight     n      =  0).      Suppose,    moreover, 

P 

we  were  to  use  (11)  to  calculate  a  revised  profit  figure     R    (X    )     for  this 

q    p 

old  proposal    X      .      It  can  be  proved  without  too  much  difficulty     (see 
P 

e.  g.  ,    Dantzig,    [  1   ]  )  that  we  must  have     R    (X    )  =     II       .      In  other  words, 

q     p  1 

every  previous  division  1  solution  which  has  not  been  rejected  by  the 
executive  program  calculation  must  offer  the  same  revised  profit 
figure  to  the  division  and  each  such  profit  figure  must  be  equal  to  the 
division  1  marginal  yield,    II     .  We  see  at  once  that  since  the  division's 

new  proposal,      X     ,      has  been  calculated  to  maximize  the  revised 

q 

revenue    R    (X)     ,      we  must  have     R    (X    )  >  R    (X    )  =  n     .      Thus,    the 

q  q    p  -    q    p        i 
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division's  (optimal)  new  proposal  must  offer  at  least  as  high  a  revised 
revenue  yield  as  its  previous  proposals,    and  only  if  it  offers  a  higher 
return  than  the  previous  proposals,    i.  e.  ,    only  if    R    (X    )  >  IT       ,      will 

q    q        i 

it  pay  to  introduce  the  new  proposal  into  the  averaged  company  solution 
X       as  given  by  (4). 

q 

8.      The  Optimal   Weighted  Average  Programs  and  Decentralization. 

Once  the  final  stage  of  the  calculation  is  reached,    an  optimal 

o  o 

set  of  weights,      u,       and    v      is  calculated  with  the  aid  of  the  executive 
6  rs  t 

program.      Substitution  of  these  optimal  weights  into  (4)  yields  the 
optimal  weighted  average  divisional  solutions.      Those  optimally  aver- 
aged decisions  constitute  the  solution  which  is  most  profitable  for  the 
company  as  a  whole. 

It  is  to  be  emphasized  that  now  division  managers  must  be 
told  by  the  company  what  weights  they  are  to  employ,    i.  e.  ,    what  com- 
bination of  their  proposals  the  company  desires  them  to  produce.      There 
is  no  automatic  motivation  mechanism  which  will  lead  division  man- 
agers to  arrive  at  such  a  combination  of  outputs  of  their  own  volition. 
In  this  way,    the  decentralization  permitted  by  decomposition  breaks 
down  completely  at  this  point. 
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To  understand  the  nature  of  the  situation  we  can  profitably 
examine  the  problem  graphically.      Figure  1  represents  the  feasib 
region  of  one  of  the  divisions.      Now,    as  is  normal  in  linear  programm- 
ing,   we  look  among  the  corners     0,    A,    B,    C,    D  and  E,    for  a  program 
which  is  optimal  from  the  point  of  view  of  the  division.      But  there  is 
no  reason  why  the   solution  which  is  optimal  for  the  company  should 
occur  at  a  corner  of  the  division's  feasible  region.      Indeed,    usually 
the  company's  optimal  solution  point  will  occur  at  a  non  corner  point 
(such  as    S)     on  the  boundary  of  the  divisional  feasible  region.      That 
is  precisely  why  a  weighted  average  of  several  divisional  proposals 
is  required  in  this  solution.     In  the  present  case  the  optimal  point,     S  , 
is  a  weighted  average  of  the  solutions  represented  by  points     D    and 
C  . 

Now,  each  revision  of  the  divisional  profit  figures  by  the  master 
program  produces  a  shift  in  iso  profit  lines  in  the  diagram.      For 
example,    at  some  stage  in  the  calculations  the  revised  profit  function 
may  be  represented  by  the  dotted  parallel  lines  in  Figure  1,    so  that 
the  division's  optimal  program  is  point    C  .     Subsequent  calculations 
will  twist  these  iso  profit  curves  about.      In  particular,    if  the  optimal 
solution  lies  at  an  interior  point  of  the  feasible  region  boundary,    such 
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as    S  ,     the  terminal  iso  profit  lines  must  be  parallel  to  the  boundary 
segment    CD    which  contains    S  .      We  know  that  this  must  be   so  from 

our  previous  observation  that  every  proposed  divisional  proposal,      X 

I 

which  is  averaged  into  the  final  solution  must  yield  the  same  revised 
profit  figure    R    (X    )  =     IT     .      In  the  case   shown  in  the  diagram,    since 
S    is  obtained  as  a  weighted  average  of    C    and    D  ,    both  solution  point 
C     and  solution  point    D    must  yield  a  revised  profit  figure  equal  to     II    ; 
i.e.,      C    and    D    must  lie  on  the  same  iso  profit  curve. 

This  tells  us  directly  why  the  price  system  can  not  be  relied 
on  to  induce  the  divisions  of  their  own  volition  to  produce  the  (company) 
optimal  solution,     S  .      Every  point  on  boundary  segment    CD  ,    including 
C,    S  and  D     must  be  equally  profitable  from  the  point  of  view  of  the 
division.      In  fact,    since    S  is  not  a  corner  of  the  feasible  region  no  set 
of  fixed  prices  can  make    S    the  most  profitable  output  for  the  division. 
For  consider  any  straight  iso  profit  line  (such  as  PP')     through    S 
which  does  not  also  go  through  points     C     and    D     .      Then  either     C 
or    D    must  lie  above     PP'    ,      i.e.,    with  the   set  of  prices  which  yield 
profit  line     PP'     either     C     or    D    must  be  more  profitable  than    S  . 

We  may  conjecture  that  this  difficulty  is,    in  part,    a  conse- 
quence of  the  linearity  of  our  problem.      If  the  boundary  of  the  division's 
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(convex)  feasible  set  of  solutions  were  curved  throughout,      i.  e.  ,    if  dim- 
inishing returns  (diminishing  technical  marginal  rates  of  substitution) 
were  everywhere  present,    this  problem  would  sometimes  disappear.      In 


Figure  2  we  see  such 

P 
X„ 


A         X, 


X. 


X, 


Figure  1 


Figure  2 


a  feasible  region  and  note  that  the  iso  profit  line     TT  !,    which  is  tan- 
gent to  optimal  point    S  ,      does  the  trick.       It  causes     S    to  be  the  output 
combination  which  is  most  profitable  for  the  division  as  well  as  for  the 
company  and  so,    with  the  prices  corresponding  to     TT'     the  divisions 
can  be  left  to  make  output  decisions  entirely  by  themselves.      Unhappily, 
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things  do  not  always  work  out  so  neatly  even  in  the  diminishing  returns 
case.     For  in  such  circumstances  the  optimal  solution  may  well  turn 

out  to  be  a  point  such  as     V     in  Figure  2,      which  is  in  the  interior  of  I 

17/ 
division!  s  feasible  region,  and  there  is  obviously,    no  iso  profit  line 

which  goes  only  through  point    V     .      However,    it  is  fortunate  that  various 

types  of  nonlinearity  can,    in  fact,    be  introduced  into  the  problem  with- 

18/ 
out  doing  serious  damage  to  the  decomposition  process.  We  will 

see  in  the  next  section  that  this  is  extremely  helpful  for  our  economic 

interpretation. 


17/    One  may  conjecture  that,    in  most  long  run  economic  problems,    both 
diminishing  returns  and  boundary  solutions  are  likely  to  be  the  rule 
rather  than  the  exception.     After  all,    diminishing  returns  are  part 
and  parcel  of  the  adaptation  of  facilities  to  varying  situations,    while 
an  inferior  optimal  solution  means  there  must  be  excess  capacities 
in  the  divisions  which,    in  the  long  run,    would  surely  be  eliminated. 

18/    Of  course,    these  nonlinearities  must  satisfy  the  appropriate  convexity 

(diminishing  returns)  requirements.      Moreover,    nonlinearity  means 

that  we  can  never  be   sure  the  algorithm  will  arrive  at  the  precise 

optimal  solution  in  a  finite  number  of  steps.     However,    we  can  be 

assured  of  its  convergence  to  the  solution  of  the  problem 
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9.      Externalities  and  Nonlinear  Divisional  Constraints 

It  must  be  admitted  that  the  linearity  assumptions  which  have 
so  far  been  employed  in  our  model  considerably  restrict  its  domain  of 
applicability.      Especially  the  type  of  external  economy  (and  to  some 
degree,    the  type  of  external  diseconomy)  which  occurs  in  the  decomposition 
structure  may  well  be  considered  a  relatively  bloodless  member  of  the 
family.      For  it  consists  only  of  cases    where  one  division's  expanded  act- 
ivity directly  reduces  the  availability  of  scarce  resources  to  other 
economic   subunits.      This  case  is  slightly  more  general  than  it  may  at 
first  appear,    for  reduced  availability  of  resources  can  affect  the  profit- 
ability of  a  division  by  forcing  it  to  adopt  a  changed  technology  (the     X.'s 
corresponding  to  outputs  produced  by  some  method    A     may  be  reduced 
to  zero,    and,    instead,    the  outputs  of  some  alternative  process,      B     may 
be  forced  into  the  next  proposed  divisional  solution  as  a  result  of 
changed  input  availabilities).      Thus  the  externalities  which  arise  in 
our  linear  model  may  not  be  so  very  different  from  the  technological 
external  economies  or  diseconomies  of  the  literature  of  welfare  economics. 
They  are  not  just  the  pecuniary  external  diseconomies  which  arise  when 
one  producer's  demand  for  an  input  bids  up  its  price  and  thereby  in- 
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19/ 
creases  the  costs  of  other  producing  units. 

iNevertheless,    it  is  desirable  to  be  able  to  deal  with  a  far 

more  general  class  of  externalities,    and,    particularly   in  th<     pn 

of  nonlinearities  in  the  divisional  constraints,  it  is  easy  to  do  so  by 

means  of  an  extremely  clever  device  invented  by    Whinston  (see  [Uj 

Chapter  VI,    pp.    36-49). 


Suppose  we  have  the  following  decomposable  program 


/ 


Max    II  =  P.X.  +  •  •  •  +  P    X      +P       ,Y,  +  •  '  •  +  P         Y 

11  ww  w+1     1  w+r     r 

subject  to  the  corporate  constraints 


aX.  +  •  ■  •    +  a.     X    +  a  .        J,     + 
11     1  lw     w  1  w+1     1 


+  a,  Y      <    C, 

lw+r     r  —        1 


(12)       < 


a       X.+  •  •  •  +  a         X    +a  Y      +  •  •  ■    +  a  Y      <    C 

ml     1  mw    w      mw+1     1  mw+r   r  —        m 

and  the  (convex  nonlinear)  divisional  constraints 


f(X  ,  .  ...X    )  <    c 
11  w    —        m+1 


f  (X     .  ...X    )  <  c 

si  w    —      m+s 


L 


g  (Y   ,  .  ..  ,  Y    .-  X    )  <  C     ^      . 
11  r       k    —      m+s  +  1 


g  (Yn,  .  .  .  ,  Y    )  <    C 

6t      1  r    -        m+s+t 


19/     This  distinction  which  was  first  formulated  by  Viner  [11]  is  of  some 

significance  because  pecuniary  externalities  are  really  part  and 
parcel  of  the  competitive  process  of  allocation  of  resources,    and, 
unlike  technological  externalities,    they  do  not  prevent  a  competitive 
general  equilibrium  from  being       Pareto    optimal. 


,i  nd 
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This  differs  from  our  earlier  model  in  two  significant  respects: 

1.  the  divisional  constraints     f.(X_,  .  .  .  ,  X    )  <    C 

i      1  w    —        i 

and    g.(Y   ,  .  .  .  ,  Y    )  <    C.     are  not  necessarily  linear, 
i      1  r    -       j 

2.  in  the  first  constraint  of  division  2, 

g    (Y   ,  .  .  .  ,  Y    , X    )  <  C  ,   ,     there  is  present  a 

2       1  r       k    —      m+s  +  1 

variable,      X     ,      representing  a  division  1  output. 


Let  us  discuss  the  significance  of  these  two  characteristics. 
The  nonlinearities,    in  fact,    make  no  essential  difference  to  our  decom- 
position procedure;     because  all  the  special  decomposition  calculations, 
the  executive  program,    the     II's   ,    and  the  revised  divisional  profit 
functions,    only  involve  the  objective  function  and  the  corporate  constraints. 
But  both  of  these  remain  linear  in  (12).     Hence,    provided  the  divisional 
constraints  yield  bounded  convex  feasible  regions  so  that  the  divisional 
programs  possess  unique  optimal  solutions  which  can  be  averaged  together 
by  the  executive  program,    all  remains  essentially  as  it  was  in  the  linear 
case. 

However,    the  presence  of    X,      in  a  division  2  constraint  is 

k 

of  the  utmost  significance  because  it  represents  a  perfectly  general 

20/ 
technological  externality.  For  it  means  that  a  division  1  output 

20/    In  the  discussion  which  follows,    for  simplicity  of  exposition,    we  deal 
with  only  one  division  1  output  and  one  division  2  constraint  but  the 
argument  can  obviously  be  extended  to  as  many  such  variables  and  con 
straints  as  desired,    and  to  the  case  where  division  2  variables  also 
enter  division  1  constraints. 
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decision  will  produce  a  shift  in  one  of  division  2's  te<  hnologi<  a] 
relationships!     This  is  precisely  what  we  mean  by  a  technological 
externality. 

But  there  is  not  the  slightest  difficulty  in  eliminating  the     X 

K 

and  reducing  our  problem  to  another  form  which  appears  to  involve  no 
externalities.      That  is,    we  can  easily  rephrase  our  problem  in  such 
a  way  that  all  our  previous  decomposition  procedures  remain  completely 
applicable  without  further  modification.      To  accomplish  this  we  merely 
rename  our  externality  variable     X      ,    writing    X     =  X  .      The  first 

k  R 

division  2  constraint  now  becomes     g-,(Y   ,  .  .  .  ,  Y    ,  X)  <  C  and 

11  r         —      m+s+1 

we  add  to  our  set  of  corporate  constraints  the  linear     definitional 

constraint    X,     -  X  =  0   .      We  then  end  up  with  a  new  problem  which  is 
k 

exactly  the  same  as  (12)  except  that  it  now  contains  one  more  linear 
corporate  constraint.      But  in  this  rewritten  version  of  our  problem 
there  is  no  variable  which  is  common  to  a  division  1  constraint  and 
a  division  2  constraint,    as  was  true  of    X       in  (12).     In  other  words, 

K. 

2l/    The  significance  of  the  nonlinearities  for  this  discussion  is  that 

if  the     g  (Y   ,  .  .  .  ,  Y    ,  X   )     is  a  nonlinear  function,    a  change  in 

11  r        K. 

X       can  produce  just  about  any  type  of  shift  in    g    ,    whereas  in 

the  linear  case     gJY,,  .  .  .  ,  Y    ,  X,  )  =  b  Y  +  •  •  •  +  b   Y    +  b        X, 
11  r       k  11  r     r      r  +  1    k 

any  change  in  the     X       must  always  cause  a  proportionate 
parallel  shift  in  the  division  2  constraint,    and  this  is   still  a  \<:\ 
restricted  type  of  externality. 
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this  simple  transformation  has  taken  a  problem  involving  a  coi  ly 

general  technological  externality  and  transformed   it  into  a  stai 

decomposable  problein  (though  into  one  which  contains  nonlinear  d 

.    22/ 

constraints) . 

10,      Concluding  Comments 


We  have  seen  that  the  decomposition  algorithm  constitutes 

a  decision  mechanism  for  an  entire  decentralized  but  coordinated 

23/ 
economic  organization.  It  can  be  taken  to  represent  anything  from 

a  single  firm  to  an  entire  economy.      For  the  first  time  it  offers  a 
systematic  approach  to  the  planning  problems  resulting  from  the  pres- 
ence of  external  economies  and  diseconomies  (or,    in  more  general 
terms,    divergence  between  private  and  social  costs  and  benefits)  and 


22/     The  same  argument  also  shows  that  nonlinearities  in  the  corporate 
constraint  which  do  not  violate  convexity,    in  principle,    cause  no 
problems  for  decomposition.      For,    formally,    there  is  no  difference 
between  a  corporate  constraint  and  the  division  2  constraint 
gi(Yi,  .  •  .  ,  Yr,  XvJ  <  Cm+s+l  >    which  constains  variables  corres- 
ponding to  outputs  of  both  divisions!     It  appears  that  the  decomposition 
procedure  is  in  no  fundamental  way  tied  to  the  linearity  assumption 
which  is  usually  employed  in  its  description. 

23/    Of  course,    it  is  not  suggested  that  this  is  how  decentralized  decisions 
are  made  in  practice.     Gale's  model  [5]  pp.  85-93,    comes  off  far 
better  than  decomposition  analysis  as  a  representation  of  decentralized 
decision  making  under  competitive  conditions.      However,    Gale  does 
not  provide  a  pricing  and  calculating  mechanism. 


V-41 


the  consequent  limitations  which  characterize   suboptimal  dec  Lsions 
made  independently  by  subdivisons  of  the  economic  unit  in  question. 

As  has  occurred  on  several  other  occasions  in  the  history  oi 
our  subject,    the  new  analysis  serves  largely  to  confirm  the  economists' 
less  rigorously  derived  conclusions.      It  reaffirms  that  the  way  to  deal 
with  externalities  is  to  modify  the  pricing  system,    offering  bonuses  in 
the  case  of  external  economies  and  imposing  penalties  on  the  producers 
of  external  diseconomies.      But  the  provisional  dual  price  construct 
provides  the  first  effective  means  whereby  these  penalties  and  bonuses 
can  be  calculated  and  imposed,    and  the  decomposition  algorithm  offers 
an  ingenious  method  for  determining  when  the  divisional  programs 
have  been  properly  coordinated  to  yield  a  solution  which  is  optimal  for 
the  economic  unit  as  a  whole. 
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APPENDIX:    DERIVATION  OF  THE  PROVISIONAL 

247 

DUAL  PRICING  THEOREMS    '    ' 


Before  turning  to  the  derivation  of  the  provisional  dual    pri 
theorems  it  is  convenient  to  deal  with  some  preliminary  matters,    in 
particular,    obtaining  expressions  for  the  values  of  our  primal  variables, 
and  for  the  changes  in  these  values  when  another  item  is  introduced  into 
the  basis. 

Suppose  we  are  given  the  following  general  linear  program 
(which  is  taken  to  include  slack  variables  among  the  variables,      X.   , 
so  that  all  of  the   structural  constraints  are  equations)  : 

(1)  Max    P  =  P  X    +  •  •  •    +  P  X 

1     1  m+n     m+n 

subject  to 


(2) 


a  nX    +  •  •  •    +  a,  X  =  C, 

11    1  lm+n    m+n  1 


a     nX,  +  •  •  •  +  a  X  =  C 

ml    1  mm+n    m+n  m 


all    X.  >    0   . 
J  - 

24/     We  emphasize  again  that  there  is  nothing  new  or  original  in  the 

materials  which  follow.      This  appendix  is  merely  intended  to  supply 
materials  which  do  not  appear  to  have  been  developed  systematically 
anywhere  in  the  literature.      The  specialist  reader  will  no  doubt 
have  the  feeling  that  he  has  seen  all  of  this  before,    but,    apparently, 
the  results  have  usually  been  absorbed  and  consolidated  in  his  mind 
out  of  a  variety  of  sources. 
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We  may  assume,    without  loss  of  generality,    that  in  the  >  urrent 

trial  solution  the  basic  (nonzero  valued)  variables  are     X. ,  .  .  .  ,  X       .       I  i 

1  m 

we  have     X.  >  0,  .  .  .  ,  X       >  0  ,      and    X         =  •  •  •  =  X  =  0  .      Since  i 

1  m  m+1  m+n 

with  zeros  in  them  drop  out,    our  equations  (2)   reduce  to 

a    X    +  ..  -    +  a      X       =  C. 
11    1  lm    m  1 


a     .X,  +  •  •  •  +  a         X       =  C 
ml    1  mm    m  m 

Using  matrix  notation,    we  can  solve  these  equations  for     X  ,    writing  them 

as 

(3)  AX  =  C,    i.  e.  ,     X  =  A_1C     , 

where    A     is  an  mxm  square  matrix  and    X    and    C     are     m    element 

column  vectors. 

Now  suppose  we  were  to  introduce  the  production  of  one  unit 

of  commodity    k    whose  output  level  was  zero  to  begin  with  (i.  e.  ,      X       is 

K. 

a  nonbasic  variable  at  this  stage).      Our  limited  resources  would  then  re- 
quire that  there  be  some  reduction  in  the  nonzero  outputs,    so  that  the 
output  of  (basic)  commodity    i    would  be  changed,    say,    from    X.     to    X  -  AX. 
In  that  case  equations  (2)  become 

a    (X     -  AX.)  +  •  •  •    +  a       (X       -  AX     )  +  a       =  C 
11      1  1  lm      m  m  Ik  1 


a     ,(X,   -  AX,)  +  •  ■  •    +  a         (X       -  AX     )  +  a         =  C 

ml      1  1  mm      m  m  mk  m 
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or  in  matrix  terms,    writing    A       lor  the  vector  of  the     k         column  of 

K. 


coefficients  in  (2) 


1.  e.  , 


A(X   -  AX)   +  A,    =  C   , 
k 


AX  -  AAX  +  A      =  C 
k 


so  that,    subtracting  (3),    we  obtain  an  expression  for     AX  , 

(4)  -AAX+A.    =0,     i.e.,     AX-A-1A,    • 

k  k 

Asa  last  preliminary  matter  let  us  write     P     for  the  ro 


w 


vector  of  the  first    m    coefficients  of  the  objective  function  (1),    i.  e.  ,    we 

-                                                                                                                 25/ 
set    P   =  [P. ,  P_,  .  .  .  ,  P     1  .       This  permits  us  to  define         the  provisional 
1       Z  m 

dual  prices  as  the  row  vector      II  -  [II  ,  II    ,  .  .  .  ,  II      ]   ,    which  is  given  by 

J-       £*  m 

(5)  n  =  pa"1  . 

This  last  expression  serves  both  as  a  definition  and  a  calculation  rule 
for  our  simplex  multipliers  (the  provisional  dual  prices).      We  have  yet  to 
show  that  these     II.     have  the  required  properties  of  dual  prices.      This 
will  be  demonstrated  with  the  aid  of  the  following  three  theorems  whose 
implications  for  duality  will  only  become  clear  in  the  discussion  which 
follows  their  derivation: 


25/     We  are  here  assuming  that  the  first    m    variables,    Xn ,  X_,  .  .  .  ,  X 

u  m 

are  the  currently  basic  variables,    i.  e.  ,    in  the  current  basic   solution 

these  are  the     X.  /  0  .      This  involves  no  loss  of  generality. 
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Theorem  1 :      For  any  commodity    k    which  is  not  in  the  current  basis, 

A     ,     the  change  in  profit  resulting  from  the  introduction  of  a  unit  of 

output     k    into  our  basic   solution,    is  given  by 

(6)  A     =  -IIA.    +P      ,      i.e.,      IIA,    +  A,    =  P 

k  k  k  k  k  k 

Proof:       From  our  profit  function  (1),    the  change  in  profit,      A     ,    resulting 

from  the  production  of  one  unit  of    k  ,      and  the  consequent  changes, 

A.,  A_,  .  .  .  ,  A       ,    in  the  other  outputs  is 
1       2  m  c 

A     =   -P.  AX.    -  P,AX,    -   ....  P     AX       +  P,      , 
k  112        2  mmk 

or,    in  matrix  notation,  A,    =   -PAX  +  P 

k  k 

so  that  from  (4)  and  (5)  A,    =   -PA "   A,    +  P     =   -IIA      +  P      . 

k  k  k  k  k 

Q.  E.  D. 
Theorem  2 :        For  any  commodity    j     in  the  current  basis 

IIA.  =  P.  . 

J  J 

Proof:  IIA  =  PA "  A  =  P      or     IIA.  =  P.     . 

J  J 

Q.  E.D. 

Theorem  3:  IIC  =  P    . 

-    -1 
Proof:       nc  =  PA      C     so  that  by  (3)  and  the  profit  equation  (1) 

nc  =  PA"  C  =  PX  =  P    . 

Q.  E.D. 

These  three  theorems  imply  all  of  the   standard  duality  results: 

i.      Theorem  1  tells  us  at  once  that  the     A       may  be  interpreted 

as  the  profit  contributed  by  introduction  of  a  unit  of  output    k  ,      i.  e.  ,    it 
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26/ 
is  the  opportunity  cost  of  not  producing  that  item    . 

ii.      Equation  (6)  of  Theorem  1  may  be  written  out  as 

a     n     +  •  •  •    +  a     ,  n       +     A,    =  P 
Ik     1  mk     m  k  k 

which  will  be  recognized  as  the     k         constraint  of  the  dual  problem,    so 
that  the     IT.     and  the     A.     satisfy  these  constraints,    with  the     A.     playing 
the  role  of  slack  variables  and  the     n.     serving  as  the  (provisional) 
dual  variables. 

iii.      Written  out,    the  equation  of  Theorem  3  is 

(7)  p  =  c,  n   +  •  •  •  +  c    n 

11  mm 

which  tells  us  that  these  dual  prices  are  exactly  large  enough  to  impute 

all  profits  to  the  firm's  scarce  inputs,    which  is,    of  course,    one  of  the  basic 

theorems  of  duality  theory. 

iv.      Comparison  of  Theorem  2  with  Theorem  1  gives  us 

A.  =  0    for  all    j     in  the  basis. 
J 

v.      Our  interpretation  of  the     II.     as  marginal  profit  figures 
can  be  justified  with  the  aid  of  Theorem  3  or  equation  (7)  from  which  we 
infer  by  direct  differentiation  that     II.   -    3P/3C.     ,    so  that     II.     is,    indeed, 


26/    There  is  a  minor  difficulty  here  about  the  sign  of     A  Usually  in 

duality  theory,    it  is  customary  to  define  the  dual  slack  variable,      L      , 
as  the  opportunity  cost  of  producing  commodity    k  ,    whereas  here 
we  have  found  it  convenient  to  interpret    A       as  the  opportunity  cost 
of  not  producing    k  .      Hence  we  would  expect    L       and    A       to  be  equal 
and  opposite  in  sign,    and,    indeed,    the  optimal  value  of    A     =   -L      . 
Moreover,    this  observation  explains  why     A       does  not  appear     with 
the  minus  sign  usual  for  the  slack  variable  in  our  dual  constraint. 


V-47 

27/ 

the  marginal  profitability  of  input    i   . 

vi.     Another   standard  duality  theorem  which  has  not  yet  been 
discussed  may  be  written  as 

(8)  Q.  A.  =  0    for  all    j  and, 

J      J 


(9)  n.U.   -  0    for  all 


i 


i    i 


Here  the     U.     represent  the  slack  variables  of  the  primal  problem  and 

the     Q.     are  the  remaining  (structural)  variables.      The  first  of  these 

equations  tells  us  that  any  item  which  will  be  produced    (Q.  >  0)     must 

have  a  zero  opportunity  loss  (A.  =  0)   .      The  second  equation  states  that 

any  input  with  unused  capacity     (U.  >  0)     must  have  a  zero  marginal 

profit  value     (II.  =  0)   .      The  first  of  these  relationships,      (8),    holds  by 

iv  ,     above,    because  for  any    X.  =  Q.     in  the  basis  (i.  e.  ,    for  anv     X    >  0), 

J  J  J 

we  have     A.  =  0  ,    and  for  any  other     X     ,    say    X,     ,    we  have     X,    =  0     since 
J  k  k 

X       is  not  in  the  basis,    so  that  in  either  case  the  equation  is   satisfied. 

Equation  (9)  will  also  hold  for  our  provisional  dual  variables, 
To  see  this,    note  first  that  the  coefficient  of  any  slack  variable,      U.    ,      in 
the  objective  function  (its  unit  profit),      P.     is  zero.      That  is,    excess  cap- 
acity is  assumed  to  produce  neither  profit  nor  loss  for  the  firm.      To 

prove  that  (9)  is  satisfied  we  observe  that  if    X.  =     U      is  in  the  basis, 

i  i 

27/     The  differentiation  is  permissible  because,    by  (5),    a  ceteris  paribus 
change  in    C.  will  not  affect  the  values  of  the     UVs.     It  may  be  noted 
that,    similarly,    by  (6),    the     Aj^'s  will  retain  their  values  when  the 
C's  vary.      Since,    by  assumption,    we  continue  to  employ  the  same 
basic   solution  throughout  the  calculation  of  our  provisional  dual 
prices  only  the  magnitude  of  the  nonzero    X's  (the  basic  primal 
variables)  will  change.     As  stated  earlier,    the  differentiation  pro- 
cess implicitly  assumes  the  absence  of  degeneracy,    for  where 
degeneracy  occurs     3P/3C.     may  not  even  be  defined. 
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so  that    U.  >  0  ,     then  by  Theorem  2     ITA     -  P     =  0     for  our  slack  vari 

i  i  i 

able,    as  was  just  observed.      But  for  such  a  slack  variable    A       is  the 

i 

unit  column  vector      {00  .  .  .    010  ...    0}     with  a  unit  in  the     i         location 

and  zeros  elsewhere.      Hence  we  must  have     ITA.   =  II      and  since  we 

i  i 

just  saw  that    IIA ,   =  0  ,      our  result  follows;     II     =  0  .      That  is,    for 
i  i 

28/ 
any     U.  >  0     we  must  have     II.   =  0     so  that    (9)  will  hold. 
i  i 


28/    However,    it  is  easily  shown  by  counterexample  (and  it  also  follows 
from  the  preceding  argument)  that  if  disposal  activities  are  not  free 
say,    if  there  is  a  cost  to  handling  excess  inputs,      U.   ,      so  that 
P.  <  0  ,     then  (9)  will  no  longer  be  valid.     In  economic  terms,    of 
course,    this  is  because  in  such  a  case  excess  inputs  are  no  longer 
free  goods   -  they  carry  with  them  a  very  real  cost. 
We  also  may  note  that  the  argument  used  to  prove  (8)  can  be  em- 
ployed to  show    X.  A.  =  0     for  all    j     where  the     X.     represent  both 
output  variables     \Q  s)     and  slack  variables     (U's).      This  means 
that  we  can  also  determine   simplex  evaluators  with  the  usual  pro- 
perties for  the   slack  variables.      The  interpretation  of  such  a  slack 
variable  evaluator     A,     is  straightforward.     It  is  the  opportunity 
gain  (frequently  negative)  to  be  obtained  by  introducing  the  slack 
variable     U.     into  the  basis;    that  is,    the  gain  (loss)  per  unit  in- 
volved in  deciding  to  leave  unused  some  amount  of  input    i     . 
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CHAPTER  VI:   SOME  ILLUSTRATIONS  OF  DECOMPOSITION 

by  Philip  Wolfe 

1.   The  Decomposition  Principle 

The  decomposition  algorithm  is  a  particular  method, 
or  family  of  methods,  for  reducing  mathematical  program- 
ming problems  having  certain  structural  properties  to  a 
sequence  of  problems  of  smaller  size.   Underlying  the 
algorithms  embraced  by  this  term  is  the  decomposition 
principle,  from  which  emerges  not  only  these  particular 
algorithms  but  also  a  general  attack  on  a  wide  variety 
of  programming  problems.   In  its  broadest  terms,  the 
principle  can  be  expressed  this  way:   In  solving  the 
linear  programming  model  of  a  particular  mathematical 
programming  problem,  do  not  lay  out  all  the  data  needed 
for  the  model  in  advance,  but  generate  those  data  only 
when  they  are  called  for  in  the  computation. 

A  primitive  instance  of  the  principle  lies  even  in 
the  transition  from  the  standard  form  of  the  simplex 
method  to  the  revised  simplex  method.   In  the  standard 
form,  when  solving  the  linear  programming  problem 

Minimize   ex 

subject  to  x  >  0  ,  Ax  =  b  , 
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we  have  at  any  stage  the  complete  standard  form  tableau 

c 

satisfying  A  x  =  b~  with  x  ^  0  ,   x  basic, 
A" 

where  the  bar  denotes  the  result  of  pivoting;   c~  is  the 

set  of  reduced  costs.   We  then  perform  these  steps: 

(I)   Select  a  pivot  column   J  by  examining 
c   (e.g.,   Min .  c. ) . 
(II)   Select  a  pivot  row  I   so  as  to  keep  the 
current  basic  solution  feasible,  by  exam- 
ining T)  and  the  column  A,  . 
(Ill)   Pivot  in  the  tableau  on  the  pivot  entry  aT ,  . 

Now  let  A.  ,..,A.    be  the  m  columns  of  the  orig- 
Jl     Jm 
inal  matrix  constituting  the  basis  for  the  tableau  above 

at  this  iteration.   When  these  columns  are  augmented  by 

the  appropriate  costs,  we  may  define  an  inverse  E  by 

the  formula  (2) ;  it  will  have  the  structure  indicated. 

Its  top  row,  containing  the  negative  of  the  shadow  prices, 

has  been  singled  out  for  special  treatment. 
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(2)   E  = 


1   c .    c . 
0   a-,    a-. 

h     h 


0  a  .   a  . 


_-l 


c . 
J 


m 


1. 

J 


m 


mj 


m 


1  -nx  -n2 


0  eu  e12 


-TT 


in 


lm 


0   e  .   e  0 

ml   mz 


mm 


We  note  that  all  the  data  of  the  standard  form  tableau  can 
can  be  obtained  by  multiplying  the  original  data  of  the 
problem  by  E   ;  namely, 


m 


c       c 

=  E 
A       A 


,  or  equivalently 


c  .  =  c  .  - )     n- 

j   j  L  i=1  i 


a±j   (all  j) 


^.-1 


e . 


=1   ik  akj 


(all  i,j) 


These  formulas  show  how  to  use  the  principle  to  re- 
duce the  amount  of  calculation  performed.   The  steps  of 
the  revised  simplex  method  corresponding  to  those  above 
for  the  standard  form  are 


(I)   Calculate  all   c.   by  (3)  and  select  the 
pivot  column   J  . 
(II)   Calculate  the  coefficients   a.j   for  the 

pivot  column  using  formula  (3)  for   j  =  J  , 
and  choose  the  pivot  row   I  . 
(Ill)   Pivot  in  the  inverse   E   (so  as  to  prepare 
for  the  next  iteration)  and  modify  the  cur- 
rent solution  b  . 
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Here  we  have  omitted  calculating  the  quantities   a. , 
for  j  /=  J  ,   and  have  calculated  only  those  items  of  the 
tableau  really  needed.   The  saving  in  work  if   n  »  m-- 
that  is,  if  the  problem  has  many  more  activities  than 
constraints—will  be  considerable. 

The  successful  applications  of  the  decomposition 
principle  carry  this  idea  even  further.   Such  appli- 
cations will  be  found  where  the  data  of  the  original  linear 
programming  model  (l)--the  costs   c.   and  the  activity 
columns  A. --have  such  regularity  that  they  themselves 
need  not  all  be  specified  in  advance  numerically,  but 
can  be  generated  when  they  need  to  be  inspected,  and  in 
some  manner  which  does  not  involve  explicitly  inspecting 
all  of  them.   In  any  application,  then,  we  are  first  con- 
fronted with 

THE  SELECTION  PROBLEM: 

Given  a  set   TTn,..,rr   of  shadow  prices: 
1'   '  m  r 

to  determine,  from  among  all  the  activity 
columns  A.   and  their  associated  costs   c. 
appropriate  to  the  linear  programming  model 
(1),  numerical  values  for  a  particular 


column 
m 


A. 
J 


for  which  the  reduced  cost 


c.  -)     tt.  a.,   has  a  suitable  value. 
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It  is  this  problem  whose  clever  solution,  if  one 
can  be  found,  makes  a  successful  application  of  the  de- 
composition principle  possible.   The  rest  of  the  work-- 
the  steps  (II)  and  (III) --must  generally  be  done  in 
pretty  much  the  same  way  as  in  the  ordinary  revised 
simplex  method.   In  the  examples  which  follow,  we  will 
see  in  each  case  how  the  linear  programming  model  (1)  is 
first  constructed,  and  then  how  the  selection  problem  is 
solved. 

In  the  next  three  sections,  the  decomposition  prin- 
ciple is  illustrated  without  regard  to  special  structure 
of  the  problem.  In  effect,  the  linear  programming  model 
(1)  is  set  up  directly,  through  an  appropriate  selection 
of  activities  for  the  problem.  When  applied  to  problems 
of  special  structure,  of  course,  the  model  (1)  is  pre- 
cisely the  extremal  problem,  and  the  decomposition  algo- 
rithm for  such  problems  is  just  one  illustration  of  the 
use  of  the  principle. 

One  of  the  examples  below  is  drawn  from  the  excellent 
expository  paper  Ll]  by  Gomory,  who  points  out  that  for 
many  problems  the  decomposition  principle  is  appropriately 
applied  in  a  dual  form:   In  a  problem  having  a  very  large 
number  of  constraints,  attention  should  be  paid  to  with- 
holding most  of  the  constraints  during  its  solution, 
generating  from  time  to  time  only  those  which  are  most 
needed  to  adequately  define  the  problem.   The  dual  simplex 
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algorithm  then  becomes  the  primary  calculational  tool, 
as  the  primal  simplex  method  is  in  this  presentation. 
We  will  confine  ourselves  here  to  examples  best  posed  in 
the  primal  form. 

2.   The  Cutting-Stock  Problem 

The  cutting-stock  problem,  posed  by  Eisemann  [2]  and 
solved  via  decomposition  by  Gilmore  and  Gomory  E3l,  il- 
lustrates the  essence  of  the  decomposition  principle  with 
a  minimum  of  difficulty  in  forming  the  linear  program- 
ming model.   To  quote  El]  with  some  modifications:  "We 
assume  that  an  unlimited  stock  of  standard  lengths   L 
of  one  material  is  available  from  which  one  is  to  cut 
lengths  to  fill  a  list  of  orders.   An  order  consists  of 
a  request  for  a  number   N.   of  pieces  of  length 
p.  ,  (i=l,..,n).   The  problem  is  to  fill  the  orders  by 
cutting  up  the  stocked  lengths  so  that  the  waste--i.e., 
unused  but  cut-up  pieces--is  minimized." 

The  linear  programming  model  is  formed  as  follows: 
Let  us  list  every  possible  way  of  cutting  up  a  length 
L   into  some  collections  of  desired  lengths   p.  ,  and 
let  the   j     activity  be  one  of  these  ways;  then   a., 
is  the  number  of  pieces  of  length   p.   obtained  in  that 
cutting.   Let   x.   be  the  number  of  lengths  to  be  cut  in 
the  j     way.   Then  to  satisfy  the  demands  we  need 
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(4)     £  .  a.j  xj  >  N.   (1-1,.., m)  ,  x.    >   0   (all  j)  . 

Since  the  total  length   )    N.  of  stock  needed  is  fixed, 

the  waste  will  be  minimized  if  the  number  of  pieces  cut 

is  minimized;  so  the  objective  is  to 


L    4    J 


Minimize 

'  J 

(In  common  with  all  students  of  this  problem,  we  ignore 
the  desirability  of  integral  answers  to  this  problem. 
In  practice,  rounding  the  answers  obtained  to  the  nearest 
integer  gives  good  enough  results.) 

The  number  of  ways  in  which  a  roll  of  length  L 
can  be  cut  into  the  desired  lengths  is  likely  to  be  enor- 
mous, so  that  this  problem  does  not  allow  an  explicit 
statement  in  the  form  (4) .   To  show  that  the  decomposition 
principle  can  handle  it,  we  need  only  show  that  the 
"selection  problem"  has  a  convenient  solution.   Since 
c.  =  1  ,  the  reduced  cost  for  the  selection  problem  is 

1  -  )    tt.  a.  .  ,  where   a.  .   is  the  number  of  lengths 
L  i  i  ij  »         ij 

p.   used. 

In  order  to  obtain  the  best  selection  we  minimize  this; 
so  the  selection  problem  is  essentially 
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(5)   Maximize   £  .  ni  Yj_   for  integral   yi  >  0  ,      p.  y.  <  L 

i  i  1 

The  solution  of  this  problem,   y.  ,  yields  the  cutting 
a^.  =  y^  ,  which  constitutes  the  column  chosen  by  the  re- 
vised simplex  method  for  the  original  problem. 

The  problem  (5)  is  a  famous  one,  the  "knapsack 
problem":   the  problem  of  filling  a  container  of  capacity 
L  with  objects  occupying  space   p.   and  having  value 
tt^   so  as  to  maximize  the  value  carried.   It  is  readily 
solved  by  a  dynamic  programming  procedure,  because  it 
has  only  one  constraint. 

Gomory  reports  El]  that  problems  as  large  as   m  =  50 
have  been  solved  in  a  few  minutes  of  IBM  7090  time. 

3.   The  Multi-Commodity  Network  Flow  Problem 

The  decomposition  principle  was  used  by  Ford  and 
Fulkerson  E4]  in  solving  the  multi-commodity  network 
flow  problem  before  the  principle  was  explicitly  formu- 
lated, and  served  as  one  of  its  inspirations.   A  network 
is  a  collection  of  nodes  joined  by  arcs   C-,,..,C   ,  each 
of  which  has  an  upper  limit   b.  (i=l,..,m)   on  the  total 
flow  of  commodities  through  it.   Some  of  the  nodes  are 
sources  for  particular  commodities,  and  some  are  sinks . 
The  objective  of  the  problem  is  to  find  the  flow  of  each 
commodity  in  each  arc  such  that  the  total  commodity  flow 
is  maximized. 
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An  ordinary  linear  programming  model  could  be  con- 
structed for  this  problem  by  letting  the  flow  of  each 
commodity  through  each  arc  be  a  separate  activity,  and 
expressing  the  constraints  that  (i)  the  total  flow  in 
each  arc  is  not  greater  than  its  stipulated  value  and 
(ii)  the  total  flow  of  a  given  commodity  through  any 
node  is  equal  to  the  amount  of  that  commodity  generated 
at  that  node.   The  total  number  of  constraints  would  then 
be  (no.  of  arcs)  +  (no.  of  nodes)  •  (no.  of  commodities). 
We  may,  however,  construct  another  model,  having  only  m 
constraints,  at  the  expense  of  greatly  increasing  the 
number  of  variables--a  circumstance  which,  thanks  to  de- 
composition, does  not  dismay  us. 

Let  us  suppose  that  we  have  listed  every  route  in 
the  network—that  is,  every  chain  of  arcs  connecting  a 
source  of  a  given  commodity  with  a  sink  for  that  commod- 
ity.  Let  the  routes  be  numbered  by  j=l,..,n  ,  and  for 
each  arc   i   let   a. .  =  1   if  arc   i   is  used  by  route 

i  ,  and  let   a..  =0   otherwise.   If  x.   is  the  flow  of 

the  given  commodity  along  route   j  ,  then  the  total  flow 

n 
over  arc   i   is   )     a.,  x.  ,  and  the  constraints  on 

arc  capacity  are 
n 


)     a. .  x.  <  b.   (i=l, • • ,m)  • 


j-l 
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Subject  to  these  constraints,  we  wish  to 

n 
Max: 


imize   )      x. 


We  may  pass  immediately  to  the  selection  problem. 

Each   c.  =  1  ,  so  that  given  the  shadow  prices   ^  ,  the 
-1  m 

reduced  costs  are   1  -  )     tt  .  a .  .  ,  and  the  selection 

L   i=1  i  ij 

problem  is  to 

m 

Minimize   )     n.  a. .  ,  where,  for   i=l,..,m  , 
u   i=l  x  1J 

a.  .  =1   if   i   is  an  arc  on  some  fixed  chain 

j   connecting  a  source  of  some  commodity  to  a 
sink  for  that  commodity. 

Let   tt.   be  thought  of  as  the  length  of  the  arc   i  ;  then, 
for  a  given  commodity,  this  is  the  problem  of  finding  the 
shortest  path  connecting  any  of  its  sources  to  any  of 
its  sinks.   If  this  problem  is  solved  for  each  commodity, 
the  shortest  of  all  such  paths  can  be  chosen,  and  it  de- 
fines the  desired  activity  for  the  extremal  problem.   The 
problem  of  finding  the  shortest  path  connecting  any  of 
one  set  of  nodes  to  any  of  another  set  is  a  standard 
problem  in  network  flows  I_5,  Ch.  Ill]  which  has  many 
solutions . 
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4.   The  Nonlinear  Programming  Problem 

The  general  (continuous)  nonlinear  programming 
problem  is 

(6)  Minimize   f(x)   subject  to   g  (x)  <  0   (i=l,..,m)  , 

i 

where  we  understand  the  functions   f   and   g   of  the 
variables   x  =  (x,,..,x  )   to  have  such  reasonable  prop- 
erties as  will  assure  a  solution  of  the  problem.   The 
essential  properties  needed  for  the  success  of  most 
methods  for  nonlinear  programming  are  the  concavity  of 
the  functions   f   and  g.  . 

A  linear  model  of  this  problem  can  be  constructed  if 

1   2 
it  is  supposed  that  a  grid  of  points   x  ,x  ,  ...   is  given 

which  are  so  densely  distributed  in  the  domain  of  the 

variable   x   that  any  point  in  that  domain  may  be  closely 

approximated  by  a  convex  combination  of  points  of  the 

1   2 
grid;  that  is,  for  any  x  ,  we  may  find  numbers   s  ,s  ,... 

such  that 

(7)  x  =  Y   SJ  xJ  ,   sj>  0  (all  j)  ,  Y,       sj  =  1  . 

J  3 

A  suitable--approximate--linear  programming  model  is  then 
Minimize   y    f(x-')  s^ 

j  r       1 

(8)  subject  to   sJ>  0  (all  j)  ,       sJ  =  1  ,  and 
Y   s.  gi(x^)  <  0   for  all 


J 
i  . 
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(9) 


When  this  problem  is  solved,  the  point  x   given  by  (7) 

will  be  an  approximate  solution  to  the  original  problem 
(6). 

Any  column  from  the  model  (8)  has  the  form 


f(xJ) 
1  . 

gi_(xJ) 


Sn>J> 


Thus,  given  shadow  prices   tt  }  the  selection  problem 

here  is 

m 


Minimize   f(x)  -n   -  ) 

0    L  i-i 


where   tt   is  the  shadow  price  associated  with  the  con- 
vexity constraint,  and  where   x   is  allowed  to  assume  any 
value.   The  success  of  this  procedure  depends  heavily 
upon  how  readily  the  problem  (9)  can  be  solved.   In  the 
general  case  it  is  not  necessarily  substantially  easier 
to  solve  than  the  original  problem,  but  in  many  cases  it 
is.   The  fact  that  it  is  expressed  without  inequalities 
often  makes  classical  extremization  techniques  practical. 

The  selection  problem  is  especially  interesting  if 
the  functions   f   and  g.   are  separable  —  if  they  can  be 
written  as  sums  of  functions  of  the  single  variables 


:k   (k=l,..,p):   f(x)  -  V  f ,  (xk) 

k 


and 


Si  00 


I 


»ik 


(xj  • 
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In  this  case  the  problem  (9)  is  solved  by  solving,  for 

each  k  ,  the  problem 

m 
Minimize   fk(xR)  -  £  ^  nigik(*k)  . 

which,  being  a  single-variable  problem,  is  likely  to  be 
very  easy. 

5.   The  Transportation  Problem 

The  "classical"  transportation  problem  runs:   Given 
m   sources ,  at  each  of  which  are  available   a.  (i=l,..,m) 
units  of  a  commodity,   n   sinks,  at  each  of  which 
b.  (j=l,..,n)   units  of  the  commodity  are  demanded,  with 

)   a.  =  )   b.  ,  and  the  costs   c . .   for  transporting  one 
i   i     j  J  J 

unit  from  source   i   to  sink  j  :   Find  the  quantity  x. . 
to  be  transported  from  source   i   to  sink  j   so  as  to 
minimize  the  cost  of  transportation.   The  linear  program- 
ming problem  is 

Minimize   )      c . .  x . .   subject  to   x . .  >  0  , 

I    xij  =ai    <al1   i}  -  li  xij  =bj    (al1  J) 

The  transportation  problem  has  a  neat  formulation 
in  terms  of  the  decomposition  algorithm,  which  is  compu- 
tationally effective  when  the  problem  is  very  "rectangu- 
lar"--that  is,  when  either  m   or   n   is  small  relative 
to  the  other.   We  suppose  here  that  m   is  relatively 
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sraall.   Our  formulation  will  be  done  directly  in  terms 
of  the  decomposition  algorithm--for  the  case  of  one  sub- 
problem--rather  than,  as  in  the  previous  examples,  in 
terms  of  the  principle  itself.   Once  the  problem  has  been 
formulated  by  this  route,  it  is  easy  to  see  how  the 
principle  might  have  been  used  directly;  but  that  is  not 
very  easy  to  see  right  off. 

The  first  set  of  constraints  of  (10)  will  be  des- 
ignated as  the  transfer  constraints.   Writing   x  = 

'xll,x12 ' " * ' x21 ' x22 ' * ' ' ' 

fer  constraints  are  A  x  =  a  ,  where 


,x  -,  ,x  0,..,x   )  ,  the  trans- 
'  ml'  m2'   '  mn7  ' 


A  = 


n  cols. 

n  cols .    ... 

n  cols. 

r  1  i  ...  i 

0  0     0 

0  0     0 

0  0     0 

11  ...  1 

0  0     0 

-,0  0     0 

0  0     0 

11  ...  1 

m  blocks 

a  = 


m 


The  single  set  of  subproblem  constraints  is   B  x 


b  ,  where 


n  cols.     n  cols. 
* .  . * w 


n  cols. 

/\ 


B  = 


1  J 


m  blocks 


b  = 


b* 

n 


Note  that  it  happens  that  no  convexity  constraint  is 
needed  here  in  the  formulation  of  the  extremal  problem. 
Suppose  that  a  set  of  proposals  is  at  hand--that  is,= 
x1,  x  ,  ..   such  that   B  x   =  b  .   The  transfer  vectors 
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k   k   . 

S    X     IS 


k  r 

are   Ax.   If  the  linear  combination   ) 

K 

feasible-- that  is,  if   )    Ax   =  a  ,  or  what  is  the  same 

u   k 

thing   )    )    s   x . .  =  a.   for  all   i  --then  we  have 

U   j  U   k     1J     L 

i        i,J,k      J     k     i,j         k      j   J 

which  shows  that   )    s   =  1  . 

U   k 

The  selection  problem  turns  out  to  be  very  simple. 
Given  the  shadow  prices   tt  t    it  is  of  course 

(11)       Minimize   (c  -  ttA)x   subject  to  x  >  0  ,   Bx  =  b 


But 

(c  -  nA)  x  =  I  (c.   -  n  )  x    , 

and  the  problem  splits  up  into  n   separate  problems, 
each  of  which  has  the  form 

(12)       Minimize  )    (c..  -  tt.)  x..   subject  to 


x 


>  0  ,  )   x. .  =  b.  , 


for  each  j=l,..,n  .   Now  the  problem  (12)  is  trivial. 
Its  solution  is : 

(13)       Let   x-j..  =  b.  ,  where   c-^  -  t^  =  Mir^  (c^  -  ^  ) 

and  x. .  =0   for   i  ^  I  . 
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The  rest  of  the  calculation  is  also  quite  easy. 
Given  the  subproblem  solution   x  ,  the  transfer  vector 
Ax  just  has  the  components   )    x..  ,  and  we  are  all 

ready  for  the  simplex  step.   The  developer  of  this  pro- 
cedure for  the  transportation  problem,  Williams  [6], 
calls  the  proposals  "supply  plans."   Evidently  each  pro- 
posal is  a  particular  means  of  satisying  the  demands 
(expressed  by  Bx  =  b)   in  the  problem;  the  transfer 
vector  Ax   aggregates  these  demands  by  source  of  supply 
and  constitutes  the  total  amount  taken  from  each  source 
by  the  particular  distribution  pattern   x  .   We  can  then 
look  on  the  entire  action  of  the  decomposition  process 
here  as  finding  that  combination  of  aggregated  demands 
which  will  (1)  exactly  use  up  all  the  available  commod- 
ity and  (2)  impute  such  shadow  prices  to  each  source 
that,  by  virtue  of  (13) ,  each  consumer  is  satisfying  his 
demand  in  the  cheapest  way. 

A  computer  routine  embodying  this  algorithm  has 
been  in  use  for  several  years  by  the  Socony-Mobil  Oil 
Company.   It  is  generally  applied  to  problems  having 
around  20  sources  and  3,000  destinations.   The  routine 
has  not  been  released  to  the  public. 
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VII.     AN  EXPERIMENTAL  STUDY  OF  THE  SIM  i'l.lM.   Ml    ["» 

A.     Introduction.     In  spite  of  extensive  computal  ce, 

relatively  little  >s  known  precisely  about  the  perfc  ce  of  the  Simp!< 

Method.     All  of  the  firm  theoretical  bounds  on  the  n  imb<  i  rations 

required  to  solve  a  general  linear  program  are  much  highei    than  the  ob- 
served behavior.      Roughly  speaking,    these  bounds  assert  that  the  number 
of  iterations  grows  factorially  with  the  size  cf  the  problem  [2],   while  the 
folklore  asserts  that  it  should  grow  linearly.     At  the  same  tim<    th(  :ce 

of  a  pivot  element  (or,    in  other  terminology,    the  choice  of  the  variables 
to  enter  and  leave  the  basis),   which  has  a  crucial  effect  on  the  number 
iterations,    continues  to  be  executed  in  most  computer  codes  according  to 
Dantzig's  somewhat  ad  hoc  suggestion  in  his  original  paper. 

The  primary  purpose  of  this   study  was  the  investigation  of  the 
relative  efficiency  of  various  rules  for  choosing  a  pivot  element,    as  mea- 
sured by  the  number  of  iterations  that  result  and  by  the  real  computing 
time  in  terms  of  a  Direct  Simplex  Method  code.     Since  the  research  was 
directed  at  but  one  aspect  of  the  computation  and  the  data  are  based 
the  statistics  of  but  one  class  of  linear  programs,   no  recommendations 
are  made  for  the  revisions  of  existing  codes.      The  design  of  a  "best:i  code, 
if  such  a  term  is  meaningful,    depends  on  many  other  factors  which  will 
not  concern  us. 

To  make  this  account  self-contained,    an  exposition  of  the  Simplex 
Method  in  the  form  that  we  have  used  is  given  in  Section  2.     It  is  to  be 
noted  that  this  exposition  differs  from  others  given  in  this  course  in  that 


the  constraints  have  been  multiplied  by     -1.     The  various  t 

have  been  tested  are  defined  in  Section  3.      The  design  and  i 
statistical  experiment  are  described  in  Section  4  as  are  thi    r< 

As  an  unexpected  byproduct  of  the  study,    two  nev 
chastic  programming  have  appeared  and  are  presented  in  S<     I 
These  were  discovered  experimentally,    by  the  analysis  of  ■  tta 

collected  in  connection  with  the  computation.      The  first    .s     • 
its  proof;   the  second  remains  a  conjecture  at  this  time  although  the  sta- 
tical evidence  is  substantial,, 

B.     The  Simplex  Method.     For  the  purposes  of  this  paper,    and 
with  no  lack  of  generality,    a  linear  program  asks  for  the  maximum  val    i 
of  a  linear  form  (the  objective  function)  on  a  polyhedral  convex  set  (the 
set  of  feasible  vectors).     If  this  maximum  exists  and  if  the  set  has  e\  - 
treme  points   (basic  feasible  vectors)  then  it  is  clear  that  the  max  m    m 
is  achieved  at  an  extreme  point.     For  a  linear  program  with  at  least  one 
extreme  point,    the  Simplex  Method  examines  a  finite  sequence  of  extre 
points,    passing  from  vertex  to  vertex  along  edges  of  the  polyhedror  while 
never  decreasing  the  objective  function.     It  terminates  either  at. the  maxi- 
mum or  with  the  information  that  the  objective  function  is  unbo  inded. 

The  central  feature  of  the  algorithm  is  an  operation  (pi"   v      g) 
which  transforms  an  algebraic  description  of  the  set  of  feasible  ve  .tor? 
with  a  specified  extreme  point  into  another  algebraic  description  of  the 
set  specifying  either  the  same  or  a  neighboring  extreme  point.     T     intro- 
duce appropriate  notation,   let  us  assume  that  the  feasible  set,  with  at 
least  one  extreme  point,    is  presented  as  the  solution  set  of  a  system  of 


m+n    linear  inequalities  in  an    n-dimensional  space.     We  may  I 
coordinates    ^ x^     so  that  the  extreme  point  specified  La  tl 

the  objective  function  is     z  =  ex    +  .  .  .   +  c    x     +  t    and  the  feasible 

11  n    n 

described  by  the    m+n    inequalities 


"allXl 


...    -  a,    x       +  b.      >  0 

In   n  1      = 


•a     ,x.    -...    -  a        x     +  b        >  0 

mi   1  mn    n  m    = 


x, 


>  0 


x 


n 


>  0 


where  the  constants    b, ,  .  .  .  ,  b         are  nonnegative.      This  description  i: 
abbreviated  conveniently  in  the  Condensed  Simplex  Tableau: 

1 


-x 


V 


m 


=  z 


Notice  that  both  the  variables     x.     and  the  variables 

J 


v.  =   -a.,x, 
l  ll   1 


a.    x     +  b. 

m   n         i 


are  restricted  to  be  nonnegative. 

The  pivot  operation  of  the  Simplex  Method  effects  an  exchange 

of  the  r&les  of  some     v.     and  some    x.    in  this  tableau.     If  the  variables 

i  J 

to  be  exchanged  are    v      and    x    ,     then  this  is  accomplished  by  solving 
for    x       in  the  equation 


v     =   -a   ,x. 
P  Pl   1 


a      x 

pq  q 


a      x     +  b 
pn    n  p 


and  substituting  the  result  in  the  remaining  equations.     This  produces  the 
new  tableau: 

■x,         ...      -v  ...      -x  L 


I 


aJl      .. 

.        a' 

iq 

In 

bi 

4 

. 

. 

. 

• 

• 

• 

• 

• 

• 

• 

• 

3*     i          •  • 

pi 

a1 

pq 

a' 
pn 

b' 
P 

. 

• 

■ 

. 

• 

• 

• 

• 

• 

• 

• 

• 

a     -,     ■  . 
ml 

•           3l              ■   • 

mq 

a' 

mn 

b1 

m 

-c 


V 


m 


-c1 


n 


=   z 


The  rules  for  calculating  the  entries  in  the  new  tableau  follow: 


(II) 
(III) 


a'      =     1/a 

pq  pq 

a'  .    -      a    ./a  for     j  i  q;     b'    =  b   /a 
PJ            PJ     pq  J  -r  -h.       p         p/     pq 

a!      =   -a.    /a  for    i  4=  Pi     c'    -    -c    /a 
iq              iq'     pq  q  q'     pq 


a: 


ij 


(a.  .a       -  a.    a    .)/a         f°r    i  41  P>     J  4=  q 

v  ij  pq     iq  pj     pq 


c1.      =    (c  ,a       -c    a    .)/a         for    j  4=  q 

j  j  pq     q  pj     pq 


b! 

i 


(b.a       -  a.    b   )/a         for    i  4=  P 

1  pq      iq  p      pq 


I    t'       =    (ta       +  c   b    )/a 

v    pq      q  p"    pq 

Naturally,   the  new  tableau  can  be  interpreted  in  the  same  man> 
ner  as  the  original.     The  current  coordinates  are    x^,  .  .  .  ,  v    ,  .  .  .  ,  x^, 
the  objective  function  is 

z  =  c'x,  +  .  .  .   +  c' v     +  .  .  .   +  c'x     +  t'   , 

11  q   p  n    n 

and  the  feasible  set  is  described  by  the    m+r.    inequalities 


■a1  x       a!   v        -  ...    -  a|   x       +  b|      >  0 

11    1  lq    p  In    n  1 


■a'    ,x     -  ...    -  a1      v     -  ....  a'      x     +  b'      >  0 

ml    1  mq   p  mn   n         m    = 

Xl  -  ° 


v  >  0 

P 


x  >  0  . 

n  - 

If  the  pivot  element  is  chosen  so  that  the  new  constants    b' ,  .  .  .  ,  b'       are 

1  m 

nonnegative,    then  the  origin    x,  =  .  .  .    =  v     =   .  .  .    =  x     =   0    is  an  extreme 

1  p  n 

point  of  the  feasible  set.     If  the  pivot  element  is  chosen  so  that    t1  >  t 
then  the  objective  function  has  not  been  decreased  by  the  pivot  operation. 
If  both  conditions  are  satisfied  then  the  informal  description  of  the  Sim- 
plex Method  given  at  the  beginning  of  this  section  has  been  realized, 
namely,    the  iteration  of  the  pivot  operation  will  produce  a  sequence  of 
extreme  points  on  which  the  objective  function  is  monotone  nondecreas- 
ing. 

It  is  a  simple  matter  to  describe  necessary  and  sufficient  con- 
ditions on  the  choice  of  a  pivot  that  will  preserve  feasibility  (b!  >  0  for 
all    i)    and  not  decrease  the  objective  function     (t1  >  t) .     First,   any 

a        i  0    in  the  row  of    b     =0     may  serve  as  pivot  element;   in  this  case, 
pq  x  p 

the  program  is  said  to  be  degenerate.     Otherwise,   the  pivot  element    a 

must  be  positive  in  view  of    b     >  0    and  Rule  II  defining    b1     above. 

P  P 

Furthermore,    by  considering  Rule  IV  for    b!,     it  is  necessary  that 


b _/a       <  b./a.  for  all    i    with    a.      >  0  . 

p'      pq  =      l'     iq  iq 

Finally,    to  insure  that  Rule  IV  does  not  yield  a    t'     that  is  smaller  than 
t,     it  is  necessary  that 

c     >  0  . 

q  = 

To  summarize,    the  candidates  for  a  pivot  element  which  may 

increase  the  objective  function  can  be  found  as  follows:     Scan  the  columns 

q    for     c     >  0;     if  there  are  none  then  the  program  is  solved.      For  those 

columns    q    with    c     >  0,     among  the    a.      >  0,     minimize    b  /a.    . 

q  iq  r      iq 

there  are  no    a.      >  0    then  the  objective  function  is  unbounded.     Other - 
iq 

wise,    in  general  we  are  led  to  a  unique  choice  of  a  potential  pivot  elem< 

a         in  each  column  for  which    c     >  0.      (Note  that  either  a  minimum 
pq  q  v 

b./a.       of  zero  or  a  tie  for  the  minimum  signals  that  the  program  is 

degenerate.     While  in  theory  this  may  lead  to  the  phenomenon  of  cycling, 

in  practice  it  is  unimportant  and  any  rule  for  breaking  the  ties  is  effective. 

We  shall  ignore  such  ties  henceforth.  ) 

Thus,   there  are  at  least  as  many  legal  choices  of  pivot  as  there 

are  columns  with    c     >  0.     The  proliferation  of  possible  pivot  choices 

q 

leads  to  a  wide  variation  in  the  number  of  iterations  required  to  solve  a 
given  linear  program.      This  phenomenon  can  be  illustrated  in  very  small 
programs.     As  an  example,    consider  the  linear  program:     Maximize 
x    +  x     +  x,     subject  to    x,    +  2x2  -1-   8x„  <  10,     9x,  +  5x,  +  6x^  <  10,     Jt.  >  0, 
x?  >  0,     x.,  >  0.     The  feasible  region  for  this  program  is  graphed  in 
Figure  1  and  is  a  truncated  pyramid. 


Figure  1 


The  six  Condensed  Simplex  Tableaus  corresponding  to  the  six  extreme 
points  of  the  figure  are  as  follows  (in  each,   the  legal  pivot  elements  are 
marked  by  a  superscript  which  indicates  the  resulting  tableau): 
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Noting  the  values  of  the  objective  function  as  calculated,   the  graph  st: 
ture  of  the  legal  pivots  is  shown  in  Figure  2,  where  the  edges  of  the  feae 
ible  polyhedron  have  been  oriented  to  show  increases  in     z. 


It  is  easily  verified  that  legal  choices  of  pivots  starting  from  Tableau  0 
may  lead  to  the  following  sequences:     OE,    OAE,    OBDE,    OACDE,    OBCDE. 
Even  in  this  small  program  any  number  of  iterations  from  one  to  four 
may  be  required  for  a  solution.     With  such  variation  possible,   the  rule 
for  a  pivot  choice  is  clearly  crucial. 

C„     The  Construction  of  Pivot  Rales.     It  is  very  easy  to  con- 
struct rules  which  will  choose  a  pivot  column  and  hence  a  p:.vot  elemei.' 
The  only  requirement  that  we  have  imposed  is  that  the  rules  employ 
information  that  is  available  "locally"  on  the  polyhedron  or  in  the  tableau, 
In  this  section,  we  shall  give  precise  definitions  of  the  rules  that  were 
tested  in  the  experiment  accompanied  by  informal  commentary  to  justify 
their  inclusion. 


The  methods  that  were  investigated  differ  01 
rule  by  which  the  new  pivot  column  is  <  hosen  at  ea,  h  iterati  C 

eligible  for  choice  as  pivot  columns  are  those  fox  whii  he.         •   1,  .  . 
is  positive.     Throughout,   when  a  rule  yields  a  tie, 
column  with  the  highest  index. 

Method  a  :     Choose    j  e    {l, .  .  .  ,n}    as  the  pivot  1  o]  imi 
c.  -  max  cg  (s  -  l,...,n)    where    c       is  defined  to  range  only  ovei   P'>    i  - 
tive    cg.     This  method  introduces  that  variable  into  the  ba-  h  per- 

mits the  greatest  improvement  in  the  objective  function  per  u      t  1<  vel  oi 
the  variable  associated  with  the  column.     This  rule  was   suggested  by 
Dantzig  in  his  original  paper  on  the  Simplex  Method  and  be 

used  in  most  computer  codes.     For  reasons  which  will  appear  later 
this  sections    it  will  be  called  the  non-basic  gradient  method. 

Method  (3:     Denote  the  potential  pivot  corresponding  to  column 
j    as     a      ..      Then  choose    j    as  the  pivot  column  if 

c  .  b  c   b 

jr.  5-       ;: 

J  S 

max 


a  a 

r  .i  s         r    s 

y  s 

This  method  introduces  that  variable  into  the  basis  which  leads  to  the 
greatest  absolute  improvement  in  the  objective  function  at  that  stage, 
may  therefore  be  called  the  greatest  absolute  increment  method.     T1 
method  was  also  considered  in  Dantzig's  original  paper.     He,    and  others 
who  have  considered  it,    seem  to  have  abandoned  it  as  a  serious  poss    - 
bility  because  of  the  additional  computation  involved. 


Method    t:    Let    u  .       (j  =:  1,.  .  .  ,nj     r       1 n)  !  I   I 

t  Vi 

choice  of    j    as  pivot  column  at  the    t       iteration  causes     c       to  b< 

r 

st 
tive  at  the     (t+1)"     iteration  and  let    ut.     =0    otherwise.      Then  choose     i 

tjr 

as  the  pivot  column  at  the    t       iteration  if    c.  >  0    and 

n  n 

2    u,  .     =  min     2    u. 

,     tjr  .      tsr 

r  =  l       J  s     r=l 

Since  this  method  minimizes  at  each  iteration  the  number  of  possible 

pivot  choices  at  the  next,    it  may  be  called  the  minimum  next  choice 

method.    This  method  was  motivated  by  the  impression  that  infeasibi]   I : 

in  the  dual  program  can  be  measured  by  the  number  of  unsatisfied 

straints. 

Method   n  :     Choose    j    as  the  pivot  column  if 

+    2  4-    2 

(ct)  (ct) 


-  max 


n  2  2  n  2  2 

S    (a+)     +  (ct)  S      S    (a+  )     +  (c+) 

i=l      XJ  J  i=l      1S  s 

This  method  will  be  referred  to  as  the  modified  basic  gradient  method; 

the  first  published  work  on  this  method  was  that  of  Dickson  and 

Frederick  [31. 


M 


ethod   r\,:     Choose  j    as  the  pivot  column  if 

+  2  +  2 

(c+)  (c+) 

v    j '  =  max  s' 

n     T~         .  2  s       n      -,  ,  "2" 
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This  is  the  basic  gradient  method. 


Method   r|2:     Choose    j    as  the  pivot  column  if 
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This  rule,   which  we  will  not.  name,    is  an  obvious  mod.  r\ . , 

designed  to  simplify  the  computation. 

Method   r\  ,:     Choose    j    as  the  pivot  column  il 

(ct),  (c+) 

J   =  max * 


n  s       n 

2    a      +  1  2    a       +  1 

i      ij  .    ,      is 

1=1       J  i=l 


This  is  the  all-variable  gradient  method. 


Rules     OC  ,    t)  ,    r)  1 ,    r\  ?     and    -q        are  all  gradient  methods 
variants  of  gradient  methods,   when  this  term  is  appropriately  inter- 
preted.    First,    recall  that  legal  pivots  in  the  Simplex  Method  move 
from  extreme  point;,  to  neighboring  extreme  point  on  the  polyhedr 
of  feasible  vectors.     Every  such  point  has  associated  coordinates 

,,..., x    ;    v. .....  v     ) ,     where  the    x.  .....  x      are  called  non-ba^ 

1  n       1  rrr  In 


x 


variables  and  the    v. .  .  .  .  ,  v        are  called  basic  variables.     A  unit 

1  m 

th 
increase  in  the  level  of  the    j       non-basic  variable  changes  the  objec- 
tive function  by    c.    and  changes  the  coordinates  of  the  point  by  the 
amount     (0,  .  .  .  ,  0, 1,  0,  .  .  .  ,  0;    -a.   , .  .  .  , -a      .),     where  the    1    appear  - 
in  the    j       component.     Using  Euclidean  distance,    the  gradient  is  given 
by  the  following  formulas  which  depend  on  the  set  of  variables  chosen 
as  the  space  in  which  to  compute: 


(A)  c.  if  non-basic  variables  are  used, 

J 

c . 

(B)  ; if  basic  variables  are  used. 

V   i     1J 
c . 

(C)  — jr-r-r- — -        if  all  variables  are  used. 
+  S  a. 


After  trivial  algebraic  manipulation,    (A),    (B),    and  (C)  correspond  to 
Methods  a  ,    r\,,   and  n  ^     respectively.     Variant    r\     is  d  s<  uss<  d     n 
Dickson  and  Frederick,  while    n  ?     differs  from    n       only  by  meas 
ing  distance  by  the  sum  of  the  absolute  values   rather  th  n    by  E 
distance. 

Method  9'    Associate  with  each  row  in  the    t       tableau  a  n  .Ti- 
ber    n  (i)     and  for  each  column  a  number    n  (j)     in  the  following  manner, 
For     t  =   0     (before  the  first  iteration)     n  (i)   =  i     for     i  =  1,  .  .  .  ,m    a—'. 
n  (j)  -  m+j    for     j  =  1,  .  .  .  ,n.       Corresponding  to  the    t        iteration  du^. 

to  pivot    a         we  define 
r  re 

nt+1(i)  =  nt(i)  for     1  /  r 

r-t+1())  =  nt0)  foi"    i  i 

nHl(r)  *  nt(c) 
nt+1(c)  =  nt(r)     . 

Further  define  at  the    t       iteration,    for     1  <  j  <  n, 


f\     if    nt(j)  <  m 
t"1»j       |o    otherwise 


and 


ut    i    ^c^»      *£    c     <0,     where  i  -■  n  (q) 
1-1,  q   q  q  t 


q 

index    q  =  1, . . . ,n 
t-l,i 

^0    otherwise 
Then,   choose    j    as  the  pivot  column  if 

+  + 

Cj 

=  max 


m  m 

2    |w.        .a..  I  +  c  2     |w     ^ja.     I  +  c1 

i=i     t-i,i  ij1      j  .=1  i  t-i;i  is  i       s 

Method  9-,:    Here    w,    .  is  defined  as    u     .      c    ,     irrespect. 

^                        t-i,i  t-l,q   q' 

.     Using  this  definition.    Method  6  -,  cl 

q  2 

pivot  column  if 


of  the  sign  of    c    .     Using  this  definition,   Method  6-,  chooses     j    at    I 


=  max 


m  m 

2    |w.    .    .a..|  +  c  2    |w.    .    .a.     I  -!-  t  + 

i=1         t-l,i    ij  j  i=1    '     t-1,1    IS1  s 

Methods  0,  and  6-,  were  suggested  more  by  economic  than  by 
rigorous  mathematical  considerations.  If  we  regard  the  original  linear 
programming  problem  as  an  activity  analysis  problem,   the  following 


s  the  amo  .  ■  1 


interpretation  can  be  placed  upon  the    c.    and    a..:     c.     . 

■J  "*"  J  J 

the    i       input  used  by  a  unit  of  the    j       activity.     In  the  gradient  meth    di 

2 
we  are  therefore  adding  together  in    2  a..    quantities  expressed  in  dif- 

ferent  units  and  dividing  this  hybrid  into    c.     expressed  in  still  different 

units.     Methods    6,  and  B?  are  rules  of  thumb  devised  to  avoid  this    ' 

ficulty.     These  methods  in  effect  pretend  that  each  iteration  is  the  final 

one.     The  methods  then  select  the  "imputed"  dual  values,   i.e.  ,   the 

imputed  prices  of  the  scarce  resources  at  each  iteration  pretending  that 

that  iteration  is  the  last  one  and  weight  the    a.  .     (expressed  in  quantity 

J 


terms)  by  the  corresponding  imputed  prices.     This  accomplishes  the  -  '-in- 
version of  the  denominators  in  the  maximands  to  the  same  dollar  units  in 
which  the  numerator  is  expressed. 

D.     Design  and  Data  of  the  Experiment.     The  nature  of  the  exp< 
iments  can  be  summarized  briefly.     In  order  to  evaluate  the  statistical 
efficiency  of  the  methods  of  pivot  choice  described  in  Section  3,   a  large 
number  of  sampling  experiments  were  performed.     Some  preliminary 
results  of  these  experiments  were  reported  in  an  earlier  paper  [4].     The 
present  paper  gives  the  full  results  of  these  experiments. 

Solutions  were  computed  for  a  large  number  of  linear  programs 
of  the  form:     Maximize    c  .  x    subject  to    Ax  <  b    and    x  >  0.      (Compare 
the  formulation  of  Section  2.)     The  elements  of  the  vector     c    were  unity 
and  the  elements  in  the  vector    b    were  10,000  throughout  the  experiments. 
The  matrix    A    was  generated  in  such  a  manner  that  its  elements  were 
independently  distributed     (a)  by  the  uniform  distribution  over  the  integers 
1  to  1000  inclusive  (Range  A)  and    (b)  over  the  set  of  numbers    1/616,  2/616, 
.  .  .  ,1000/616    inclusive  (Range  B).     The  dimensions  of  the    A -matrix  con- 
sidered were    5x5,   10  x  10,   15  x  15,    20  x  20,    25  x  25.     For   each  size, 
two  independent  samples  of  100  problems  were  obtained  for  range  A 
(Tables  I  and  II)  and  one  sample  was  obtained  for  range  B  (Table  III). 
Each  individual  linear  program  was  solved  by  nine  different  methods  of 
choosing  pivots.     For  each  problem  and  method  the  number  of  iterations 
and  the  time  taken  for  the  solution  were  recorded.     The  means  of  the 

iteration  counts    MT,     the  means  of  the  time  counts  (in  sixtieths  of  a 

I 

second)    M_,,     the  standard  deviations  of    I    and    T,     and  the  ratio 


. 


MT/MT    are  displayed  in  Tables  I,   II,   and  III. 

The  results  of  Tables  I  and  II  agree  overwhelmingly. 

virtually  impossible  to  choose  among  methods     n  ,    n    ,    r,  ? ,   and  n  .     !    i 

matrices  of  any  size,    each  one  cf  them  taking  on  the  average  18  to  19 

iterations  to  solve  a  25  x  25  problem.      Ncr  is  any  obvious  rark.::g  of 

methods  achieved  if  we  consider  the  mean  time.     Next  best  is  method 

(3,     taking  on  the  average  between  22  and  23  iterations.     This  is  fo!'   iwed 

by  methods     a  ,    Q        Q  It  is  again  difficult  to  rank  these  three  metl 

amongst  themselves  on  the  basis  of  the  mean  iteration  counts.     F 

obvious  reasons     a      does  much  better  than     0     or   0  ?     as  far  as  t'.me 

is  concerned.     Last  among  the  methods  is    e     with  respect  to  both  itera- 
te 
tion  counts  and  real  time. 

Contrasting  Table  III  with  Tables  I  and  II  we  ascertain  the  fol- 
lowing interesting  facts:     (1)  the  basic  gradient  methods  and  methods     e 
and    (3     are  invariant  with  respect  to  the  range  of  the    a.  .     coeff'.v  er*?e 
In  fact,   the    M's    and  the    M's    for  these  seven  methods  remain  \    - 
tually  unchanged  as  the  range  of  the    a,,    is  severely  reduced.      (2)  O 
the  other  hand,     a,    8,  and  0       are  sensitive  to  the  range  of  coeff       ents, 
with  the  reduction  in  the  range  creating  an  improvement  in  the  relative 
efficiencies  of  these  methods.     Method   a     is  still  inferior  to    (3     in 
terms  of  average  iteration  count  but  becomes  superior  to    (3     in  terms 


It  may  be  mentioned  for  the  sake  of  completeness  that  two  other  methods 
were  also  briefly  experimented  with.     Method  &  chooses  randomly  among 
eligible  pivot  columns,   method  y    is  the  method  of  :,least  unit  ascent,1 
that  is,   we  choose    j    as  the  pivot  column  if    cT  =  rnin  c]j.     Average  itera- 
tion counts  for  25  x  25  problems  were  194.58   '    for      method    y      and 
57.20  for  method  6. 


16, 
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229.90  231.  16 

455.63 

411.44 

11.00 

13.30 

14.67 

12.  09 

13.  87 

12.23 

12.44 

12.97 

12.  44 

9.29 

4.89 

15.34 

3.  81 

3.  98 

3.73 

3.  74 

7.  16 

7.  87 

140. 39 

63.  56 

220.20 

45.44 

189.04 

44.  88 

46.41 

108.96 

97.  90 

TABLE  II.     RANGE  A 
SAMPLE  2 


Method 


Dimen- 

sion of 

or 

(3 

e 

Tl 

\ 

^2 

^3 

ei 

6£ 

Matrix 

5x5 

MX 

4.37 

3.26 

3.91 

3.28 

3.27 

3.31 

3.27 

4.  34 

4.  3  5 

Mj 

3.  04 

2.77 

4.  11 

2.44 

2.  85 

2.46 

2.  79 

3.61 

3.  44 

MT:  Mj 

.  70 

.  85 

1.  05 

.  74 

.  87 

.  74 

.  85 

.  83 

.  79 

°"I 

1.  56 

1,  07 

1.46 

1.  02 

1.  02 

1.03 

I.  02 

1.  49 

1.  51 

CT'p 

.99 

.  85 

1.26 

.  85 

.86 

.  84 

.  80 

1.  20 

:.  16 

10  x  10 

My 

9.42 

7.43 

9.62 

6.  58 

6.64 

6.66 

6.63 

9.29 

9.  10 

Mx 

18.  57 

18.98 

28.  71 

15.24 

16.  35 

15.37 

15.  55 

22.  52 

21.  51 

M-jirMj 

1.97 

2.  55 

2.98 

2.32 

2.46 

2.30 

2.  34 

2.42 

2. 

°I 

2.  85 

2.  09 

3.45 

1.51 

1.  54 

1.60 

1.  52 

2.  59 

2.  60 

°T 

5.46 

5.  03 

9.  13 

3.37 

3.  74 

3.  58 

3.  56 

6.  02 

5.  3" 

15  x  15 

Mj 

16.38 

11.92 

20.44 

10.26 

10.  31 

10.28 

10.  25 

15.  42 

14.  89 

Mrjn 

67.44 

62.  16 

119.97 

48.48 

51.66 

48.  80 

49.  56 

86.38 

71.74 

Mrp:  Mj 

4.  12 

5.21 

5.  87 

4.  72 

5.  01 

4.  74 

4.  85 

5.  60 

4.  81 

°"I 

3.  72 

2.79 

6.28 

2.41 

2.47 

2.38 

2.  30 

3.99 

3.  51 

CTrp 

18.  05 

14.  10 

35.  26 

11.  09 

12.  17 

11.  03 

10.  91 

16.  54 

20  x  20 

Mj 

23.  56 

16.75 

35.36 

14.63 

14.75 

14.  88 

14.  64 

25.46 

25.  56 

M^ 

230.65 

147.98 

345.  57 

116. 41 

127.  14 

119. 36 

119.26 

212. 08 

208.  74 

Mrp:  MT 

9.  79 

8.  83 

9.77 

7.96 

8.62 

8.  02 

8.  14 

8.  33 

8.  16 

J.              JL 

°1 

6.99 

3.71 

10.  02 

2.95 

2.  86 

2.98 

2.  82 

5.69 

5.  20 

°"T 

v 

31.  77 

29.80 

22.  79 

35.  57 

23.44 

22.  66 

47.  11 

42.  35 

25  x  25 

Mj 

33.46 

22.  58 

56.  16 

19.25 

18.  58 

18.82 

18.46 

34.  37 

23.  90 

Mrp 

407.  16 

301. 15  839. 35 

232.65 

258.45 

230. 32 

230.  36 

441. 95  421. 85 

MT:Mj 

12.  17 

13.34 

14.95 

12.  08 

13.  91 

12.23 

12.  47 

12.  85 

12.  44 

9.  01 

4.  95 

17.  92 

4.  89 

3.71 

3.  56 

3.66 

8.  24 

6.  53 

0°rp 

v 

64.62 

322. 38 

57.  52 

113. 32 

43.43 

45.44 

11?. 94 

81.  41 

The  computed  value  appears  to  be  unreasonably  large  and  is  probably  d-ue  to 
error  in  output  medium. 


18. 


TABLE 





RANGE  B 


Method 


Dimen- 
sion of 
Matrix 


a 


5x5 


MI 

3.  97 

3. 

12 

3.  57 

3. 

00 

2. 

99 

3. 

09 

3. 

05 

4. 

11 

4. 

08 

Mrp 

2.75 

2. 

72 

3.  85 

2. 

35 

2. 

67 

2. 

38 

2. 

73 

3  _ 

54 

3. 

\n 

MT: 

MI 

.69 

87 

1.  07 

78 

89 

77 

89 

g 

86 

, 

crT 

1.34 

1. 

07 

1.  55 

95 

93 

1. 

01 

# 

96 

1. 

14 

1. 

40 

(TT 

.93 

90 

1.31 

77 

84 

82 

78 

1. 

17 

i . 

10 

10  x 

10 

MT 

8.  87 

7, 

13 

10.36 

6. 

56 

6. 

63 

6. 

70 

6. 

61 

9. 

2r 

9. 

21 

M-p 

17.63 

18. 

47 

30.68 

15. 

36 

16. 

39 

15. 

66 

15. 

63 

22. 

91 

22. 

00 

Mr^: 

MT 

1.98 

2. 

59 

2.96 

2. 

34 

2. 

47 

2. 

33 

2. 

36 

2. 

44 

2. 

38 

°1 

2.  28 

1. 

73 

3.  29 

1. 

40 

1. 

48 

1. 

44 

1. 

45 

2. 

14 

2. 

16 

(TT 

4.46 

4. 

18 

9.  05 

3. 

16 

3. 

48 

3. 

23 

3. 

33 

5. 

14 

4. 

99 

15  x 

15 

Mj 

14.44 

11. 

30 

20.  38 

10. 

31 

10. 

35 

10. 

20 

10. 

15 

15. 

02 

14. 

41 

Mt 

58.  78 

59. 

57 

120.  93 

49. 

10 

52. 

28 

48. 

74 

49. 

05 

74. 

53 

^0. 

34 

MT: 

MT 

4.  07 

5. 

27 

5.  93 

4. 

76 

5. 

05 

4. 

77 

4. 

83 

4. 

96 

4. 

83 

(TT 

3.  11 

3. 

06 

6.  57 

2. 

26 

2. 

28 

2. 

32 

2. 

13 

2. 

96 

2. 

89 

12.  56 

15. 

12 

37.  10 

10. 

47 

11. 

14 

10. 

84 

10. 

12 

14. 

43 

13. 

97 

20  x 

20 

MI 

20.  61 

17. 

02 

35.  17 

14. 

20 

14. 

20 

14. 

48 

14, 

30 

20. 

50 

20. 

14 

M-t 

143. 44 

150. 

42 

344. 56 

114. 

36 

121. 

95 

117. 

58 

117. 

24 

173. 

i2 

166. 

67 

MT: 

Mr 

6,  96 

8. 

83 

9.79 

8. 

05 

8. 

58 

8. 

12 

8. 

19 

8. 

44 

8. 

27 

°1 

4.  64 

4. 

00 

10.  91 

2. 

68 

2. 

'72 

3. 

01 

2. 

97 

4. 

22 

3. 

"0 

Cq-' 

32.  14 

34, 

04 

105. 58 

20. 

75 

22. 

70 

23. 

58 

23. 

84 

35. 

31 

30. 

35 

25  x 

25 

Mr 

27.  91 

22. 

43 

56.  02 

18, 

69 

18. 

68 

18 

48 

18. 

57 

2'7. 

85 

26. 

60 

MT 

295.96 

300. 

16 

825.  18 

227. 

65 

242. 

90 

227. 

64 

231. 

47 

356 

26 

334. 

02 

M-r: 

MT 

10.60 

13. 

38 

14.  73 

12. 

18 

13. 

00 

12, 

31 

12. 

46 

12 

79 

12. 

55 

0"! 

5,  55 

4, 

57 

13.  28 

3. 

50 

3. 

63 

3 

48 

3. 

74 

5 

68 

4. 

53 

CTrp 

58.  77 

59. 

84 

191. 02 

42. 

82 

46. 

73 

42 

26 

46. 

33 

71 

74 

56 

■Q 

of  average  time.     Methods     61  and  0^    appear  roughly  equivalent  to    a 
terms  of    M,    but  are  distinctly  inferior  in  terms  of    M 

A  heuristic  explanation  of  thus  phenomenon  for  method    a     may 
be  given  in  the  following  terms,      Coi  is    let  two  succes.-    ve  at 

which    a         and    a  have  been  chosen  as  pivots  resp<  .  »   'fly.     A - 

all  the    a   .    to  have  been  multiplied  by  an  as  yet  unspecified  s     tli    fact 

1j 

k.     Then,   by  the  rules  for  transforming  any  tableau  into  a  rew  one,   w^ 

observe  that  those  elements     c!    which  are  not  in  the  pivot  columi  -         i  .  , 

J 

j  /  q)    are  unaffected  by  the  scale  factor,     c1     however  is     -c    /ka         ai  d 

q  q         cq 

is  increased  in  absolute  value  if    k  <1,     i.  e.   if  the  ranee  of  the    a 

ij 

comDressed.     Finally  the  new    a!,    are  unaffected  by    k    except  for    a' 

ij  ^  Pq 

which  is  altered  as  a  result  of  the  change  in  scale  in  the  same  propor- 
tion as  is    c'  .     Now  if  in  this  new  tableau  the  new  pivot  is  not  in  row    p, 

q  c 

then    c"    is  more  negative  or  less  positive  for    k  <1    than  it  would  be  for 
k  =  1.     If  the  new  pivot  is  in  row    p    it  is  not  less  negative  or  is  t  5t  m  >i  < 
positive  for    k  <1    than  for    k  =  1.     This  means  that  once  a    c      becomes 
negative  it  is  less  likely  that  during  subsequent  iterations  it  will  become. 
positive  again;   in  other  words,    once  we  have  eliminated  an  a-.T  v  *\-  i:    m 
the  basis,    it  is  less  likely  with    k  ^.  1    than  with    k  =  1    that  this  act:  Taty 
will  be  reintroduced  into  the  basis.     This  provides  a  saving  in  iterat    >    - 
since  it  is  wasteful  from  this  point  of  view  to  eliminate  at  an  intermediate 
stage  an  activity  from  the  basis  which  will  eventually  be  reintroduced. 

E.     Two  Theorems  on  Stochastic  Programming.     In  additicn  to 
the  primary  data  of  the  experiment  reported  earlier  in  this  paper  certa.' 
other  data  were  collected  out  of  sheer  curiosity.     No  specifi "      se  waf 


planned  for  these  numbers;    however,    they  displayed  an  unexpe<  ted   r< 
larity  which  has  led  us  to  formulate  two  results  in  stochastic  program- 
ming.     The  first  will  be  presented  with  its  proof,   while  the  second 
remains  a  conjecture. 

Every  program  solved  has  an  associated  value:     the  maximum 
achieved  by  the  objective  function.     With  no  prior  judgments  as  to  the 
distribution  of  these  values,    they  were  tabulated.     Surprisingly,    for 
large  problems,   they  clustered  very  close  to    20.     After  the  fact,   th  s 
discovery  found  a  ready  explanation.     The    a.,    used  ranged  from    1    to 
1000    with  average  about    500.     Hence  the  average  constraint  is  about 
2  500  x.  <  10,  000    which  yields  a  maximum  for     2  x.     of    20.     Now  that 
the  observed  phenomenon  is  "obvious''  it  only  remains  to  prove  it. 

To  place  the  problem  in  a  slightly  more  general  context,    con- 
sider the  class  of  linear  programs  of  the  form:     Maximize    xn  +  .  .  .  -J-  x 

to  In 

subject  to     2   a.  .x.  <  b    and    x.  >  0    for     i    and     j  =  1,  .  .  .  ,  n.     We  shall 

assume  that  the  coefficients     a.  .    are  values  of  independent  random 

variables  all  identically  and  uniformly  distributed  on  the  interval    [r,  s] 

where    0  <r  <  s.     We  shall  also  assume  that    b    is  a  positive  constant. 

Thus,    for  each    n,     the  maximizing  values  of  the  objective  function 

define  a  random  variable     Z       which  depends  on  the  parameters     r,    s, 

n 

and    b. 

THEOREM  1. 

lim  E(ZJ  = 


n'       r  +  s 
n-*  °c 

lim        <r2  (Z    )  =      0 

v    n 
n—oo 


(Note  that,   when  all  coefficients    a.,    assume  the.      -Mean     „' 

~^~  ,     the  objective  function  has  the  maximum  value    — --        He 

c  r+s 

theorem  asserts  that  the  mean  value  of  a  program  converges  to  the 
value  of  the  mean  program.  ) 

PROOF.     The  class  of    n  by  n    matrix  game  with  pa 
a.  . 
matrices     (-M    is  closely  related  to  our  linear  programs.     As     - 

known,    if    z    is  the  maximum  value  of  the  objective  function  in  the  1 

1 

ear  program  then    v  =  —     is  the  value  of  the  matrix  game.      Thus,    for 

tu 

each    n,     the  values  of  the  matrix  games  define  a  random  variable 
which  satisfies 

(1)  V     = 


! 


n       Z 
n 


LEMMA     1, 


v    n'        2b 
PROOF.     The  truth  of  this  assertion  follows  immediately  *•   >m 

- 

the  simple  observation  that    V       is  symmetrically  distributed  abo  :t   ~ —  . 

n  2t. 

Formally,   the  payoff  entries  of  the  class  of  games  with  matrices 

(2a. . -r-s  "\ 
- — A       --  J      are  chosen  independently  from  identical  uniform  d   i 

r      s  =i        c  -  r  - 
tributions  on  the  interval      -  \~  ,    ~?-~  I  .     If    M    is  in  this  class  then  so 

L       2b  2b  J 

T  T 

is     -M        and    v(M)   -    -v(-M    )„     The  assertion  of  the  lemma  follows  by 

observing  that  the  distribution  of    V       differs  from  the  distribution  of  the 
6  n 

values  of  games  with  matrix    M    only  in  terms  of  a  translation  by  a- 

r  a*  g 
amount     ^,     .  Q.  Z.  D. 


In  view  of  equation  (1),   we  may  now  conclude  that 

E^Zr)  >  E(V    )  "  FTs   '      since  in  general     E(^-)  >  -j™,     for  positive  random 
variables    X,     with  equality  holding  if  and  only  if    X     is  constant.     How- 
ever,   in  view  of  the  following  lemma,    the  limiting  statements  of  the 
theorem  will  be  validated  if  it  can  be  shown  that 

(2)  lim     cr2(V    )  =   0  . 

n—  oo 

LEMMA  2.     Let    V       be  a  sequence  of  random  variables  satis - 

n  n 

fying     0  <  a  <  V     <  (3        and  with  identical  means    m.     If     lim     a    (V    )  =   0 

n—  oo 
then 

(3)  lim     E(^-)    =  — 

n— -  oo  n 

PROOF:  If  m  =  a  or  m  =  p  then  <r(V  )  =  0  for  all  n  and 
the  lemma  is  trivial.  Otherwise  a  <  m  <  (3  and,  by  hypothesis,  for  n 
large  enough 

min{m-a  ,   6-m}  >  cr(V    )  =  <r 

For  these    n:     if    a     >  0,     choose  constants    h     >  0    so  that 

n  n 

mini  m-CK  ,  6  -m}  =  h   cr    ;     if    cr     =0,      set    h     =  n.      Clearly    h     >  1, 

1  '  r         '         n   n  n  n  n  = 

lim         h     =  oc      and     lim      ho-     =0. 

n  _       n  n 

n-*-  oo  n-*  oo 

By  Ghebyshev's  Inequality,    for  any    h  >  0, 

(4)  Probfm  -  her     <   V     <  m  +  he-  }  ">  1 

v   '  l  n  =       n  =      ■■ nJ    =  ,2 

h 

The  definition  of    h       insures 

n 

a   <  m  -  h   cr     <m+ho-     <  (3    . 
=  nn=  nn  =  r 


23, 


Using  the  central  interval  and  (4),   we  have 


\ 


(5)  f   1-411  =t^UeM 


h2 

n 


\ 


m+  ha     I    s    •"-'  I V 

V  n    n/  \    n 


Using  (4)  on  the  outer  pair  of  intervals  and  a  trivial  bound  on  the  central 
interval,   we  have 

n  n   n       a  h 

n 

Since  (5)  and  (6)  are  valid  for  all    n     sufficiently  large, 

-L  <     am    E(~)  <    — 

m    =  _       VV     '  =     m 

n-*  oo  n 

and  the  lemma  is  proved. 

We  now  return  to    V       as  the  random  variable  defined  bv  the 

n  ' 

a.  . 

values  of  the    n  by  n    matrix  games     (-r-^-)   . 

LEMMA  3. 

lim        o-2(V    )  =  0  . 
n— ►  °c 

PROOF.     This  lemma  will  be  established  by  showing,    for 

arbitrary    e  >  0, 

lim     Prob{m-e  <  V     <  m+e  }  -  1 

~.  =      n  = 

n-^  oo 

where    m  =     ?,      -  E(V   ).     Since    V       is  symmetric  about    m,     this  is 

equivalent  to  showing: 

(7)  lim      Prob{V     <  m+e  }  =  1  . 

n—  oo 

Suppose  Player  2  uses  the  mixed  strategy  (— ,  ...,—)  which 
places  equal  weights  on  each  of  his  pure  strategies.  Then  he  will  lose 
no  more  than 


24, 


a..  +  .  .  .  +  a. 

(8)  U     =  max  -li i£   . 

W  n  .  nb  * 

If    V       is  the  value  of  the  game,    then  trivially    V     <  U    .     Therefore,    if 

n  °  7        n  -      n 

a  random  variable    U       is  defined  by  (8)  then  to  show   (7)  it  is  sufficient 


to  show  that 


Finally,    let 


lim    ProbfU     <  m+e}=  1  . 

n  =  * 

n-*oo 


X, +  ...■+  X 

S    =-i Si 

n  n 


where  the    X.     are  independent  random  variables  identically  and  uni - 

J 

r      s 
formly  distributed  on    r,  r     and  define 


Then 


p     =  Prob{S     <  m+e}  . 

*n  n  = 


ProblU     <  m+e  }  =  pn 


and  hence  it  will  be  sufficient  to  show 


lim    p     =  1  . 
n-*oo 


After  the  change  of  variables'     Y.  =  (X.  -  m)/(s  -r), 

1  1 

f  Y.  +  .  . .  +  Y 

1  n     ^        e 


p     =  Prob  I  - —  < 

*n  n  =     s  -  r 

4      r  1/2  p  1/2 


,  ,4      pl/<5  pi/<s  4 

-1/2  -1/2 


n    e 


"''Our  original  proof,  which  was  based  on  an  appeal  to  the  Central  Limit 
Theorem,   has  been  simplified  by  a  suggestion  of  David  Cantor. 


25, 


y  ,4      pl/2  pl/2 

=  l-iLrt-     J  •••     J  (Syf  +  6    Eyfyf)dy  dy 

n  £  -1/2  -1/2      x  1<J 

>1       (g  ". r)4 

n  e 

since  the  explicit  value  of  the  last  definite  integral  is  less  than    n   . 
Therefore, 

*n  =  4 

ne 

Since    p     <  1,       lim    p     =  1    and  the  lemma  is  proved, 
n— *-oo 
To  recapitulate,   Lemma  3  establishes  that,    as    n—  oo  ,     the 

variance  of  the  distribution  of  the  game  valuestends  to  zero.     Lemma 

2  then  asserts  that  the  expected  value  of  the  programs  tends  to  the 

reciprocal  of  the  mean  of  the  game  values.     However,   this  has  been 

r  +  s 

evaluated  as   — in  Lemma  1  and  therefore 

2b 

lim    E(Z    )  =  -Q-   . 

«,  n        r  + s 

n— *■  oo 

The  fact  that      lim   or    (Z    )  =  0    is  an  immediate  consequence  of  Lemma 
II-*"  oo 

3  and  the  fact  that     V     =  -^—    is  a  positive  random  variable,   bounded 

n       £l 
n 

away  from  zero.     This  completes  the  proof  of  the  theorem. 

The  second  theorem  in  stochastic  programming  was  also  sug- 
gested by  the  data  that  were  generated  in  the  course  of  the  experiments. 
A  count  was  kept  of  the  number  of  slack  variables  appearing  in  the  solu- 
tion, that  is,  the  number  of  constraints  which  are  satisfied  as  strict 
inequalities  at  the  maximum.     Denoting  the  number  of  slack  variables 
in  the  solution  as  a  fraction  of  the  total  number    m    of    nonzero  vari- 
ables in  the  solution  of  a  problem  with    m    constraints  and    n    variables 


26, 


by    F        ,     we  state  as  a  conjecture 

3        mn  J 

THEOREM  2.     If        lim         -JJL   =  l    then        lim  E(F        )  =  L. 

„,     m+n  v    mn' 

m ,  n—  oo  m  (  n—  oo 

We  have  no  proof  of  this  conjecture  but  a  considerable  amount 
of  statistical  evidence.      Table  IV  summarizes  this  evidence  for  prob- 
lems with    A -matrices  ranging  in  size  from     5  x  55    to     55  x  5.      Each 
cell  contains  two  numbers:     the  upper  one  is  the  empirically  calculated 

mean  fraction    F  over     100    matrices  and  the  lower  one  is  the  theo- 

mn 

retical  limit  of    m/(m+n).     Results  were  not  calculated  for  problems  for 
which    m+  n  >  60    in  order  to  save  computer  time. 

The  statistical  evidence  for  the  conjecture  appears  to  be  very 
good.     Nevertheless  the  conjecture  is  somewhat  counterintuitive.     Since 
there  are    n    activities  and    m     slack  variables,   the  conjecture  states,    in 
effect,    that  the  probabilities  with  which  the  maximization  process  selects 
the  two  types  of  variables  for  inclusion  in  the  solution  are  the  same  as  the 
proportions  in  which  they  occur;   that  is,    activities  and  slack  variables 
are  equiprobable  in  a  limiting  sense.     Since  slack  variables  make  no  con- 
tribution to  the  maximand  one  may  have  expected  intuitively  that  activities 
will  tend  to  be  favored  over  slack  variables;   that  is,    one  would  expect 

F         <  — . This  is  in  fact  true  for  52  of  the  66  cases  reported  in 

mn      m+n 

Table  IV.     In  fact,   the  expected  value    E(F22)     can  be  calculated  exactly 
and  turns  out  to  be  .333.     If  the  conjecture  is  true,    it  appears  likely 
therefore  that  convergence  will  be  from  below.     In  any  case  it  is  still 
surprising  how  good  the  agreement  is  between  the  experimental  figures 
and  the  limiting  figure    m/(m+n).     Finally,    it  is  noteworthy  that  on  the 
average  not  much  further  improvement  can  be  expected  in  the  efficiency 


TABLE  IVs 
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n  5 


10    15    20    25 


30 


35    40 


45 


50 


55 


m 
5 


10 


15 


20 


25 


30 


35 


40 


45 


50 


55 


.450  .302  .250  .196  .174  .136.120 
.500  .333  .250  .200  .16?  .1431.125 


118 
111 


092  .112  . 080 
100  .091  . 083 


638  .474  .372  .319  .293  .246 


.214 


667  .500  .400  .333  .286  .250  .222 


195 
200 


188 


.  160 

.  166 


738  . 591  . 477  . 394  . 369  . 321 
750  .600  . 500  .428  . 375  .333 


.277  |,267  .246 
. 300  . 273  .250 


792  .660  .  553 
800  ,667  .  571 


. 503  .416  .387 
. 500  .444  .400 


371  j.  330 
3641. 333 


828  . 701  .632 


. 550  .497  . 444 


833  . 714  .625  .556  . 500  .455 


.412 
.417 


856  . 753  .660  ,592  . 547  .489! 

857  . 750  .667  .600  . 545  . 500i 


U 


876    .  784    .696  j.629    .  580 
875    .778    .7001.636    . 583 


888  . 800   .732  j.669  i 

889  .  800  j.727    .667  j 


902    . 812  L  745 
900    . 818    .750 


910    . 832  ; 
909    . 833  I 


916 
917 


First  row  in  each  cell:    actual  fraction  of  slack  variables, 
Second  row  in  each  cell:    true    m/(m-fn). 
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of  the  simplex  method  by  even  better  pivot  choice  methods.     If  the  conjec- 
ture is  true,    at  least  12-13  iterations  will  be  required  on  the  average  to 
solve     25  x  25    problems  by  any  method  since  25  slack  variables  appear 
in  the  initial  basis  but  only  about  half  that  number  will  remain  in  the  final 
basis.      Method    r|     accomplishes  the  solution  in  13-19  iterations.     It  thus 
commits  on  the  average  only  3  "mistakes,  "  that  is,    it  introduces  into  the 
basis  in  the  course  of  finding  a  solution  only  three  variables  which  are 
subsequently  eliminated.     Improvement  of  course  is  possible,   particularly 
for  problems  of  large  dimensions,   but  the  efficiency  of  the  basic  gradient 
methods  is  already  very  good  in  the  light  of  Theorem  2. 

>'fi  5|c  >!<  ;j<  i'f  >',<  jjs  :\< 
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CHAPTER  VIII:   COMPUTER  ROUTINES  FOR 
LINEAR  PROGRAMMING  AND  DECOMPOSITION 

by  Philip  Wolfe 

1.   Linear  Programming  Routines 

Almost  every  computer  has  had  at  least  one  linear 
programming  routine  written  for  it.   Some  of  the  current 
routines  for  leading  computers  are  briefly  described  in 
the  table  on  page  4.   Each  of  these  routines  differs 
from  the  others  in  some  features  unconnected  with  the 
computer  for  which  it  has  been  programmed;  as  the  fol- 
lowing sections  indicate,  there  is  a  bewildering  variety 
of  ways  in  which  input,  computation,  and  output  may  be 
handled,  and  each  of  these  ways  emphasizes  a  facility 
important  to  some  linear  programming  users  at  the  ex- 
pense of  other  facilities. 

Perhaps  an  effective  analysis  of  linear  programming 
routines  can  be  made  in  terms  of  speed  (measured  by  the 
time  required  to  solve  a  problem  of  given  size) ,  problem 
size  (the  size  of  the  largest  problem  which  can  be  han- 
dled—measured commonly  only  in  terms  of  the  number  of 
constraints,  which  is  by  far  the  most  important  problem 
parameter) ,  and  flexibility  (best  measured  for  most  users 
by  the  availability  of  "postoptimal  analysis"--parametric 
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studies  and  cost  ranging--and  convenient  "system"  fea- 
tures, such  as  the  ability  to  restart  problems  after 
interruption  or  to  change  them  in  midstream) .   Three  of 
the  routines  available  for  the  IBM  7090  illustrate  well 
the  differences  that  result  from  different  emphases  of 
these  measures.   The  routine  IKLP/90  (LP90  for  short) 
can  handle  problems  having  up  to  1023  constraints,  more 
than  the  other  routines  for  this  machine  can  do  effec- 
tively.  This  of  course  requires  the  extensive  use  of 
magnetic  tape.   In  order  to  assure  solution  of  large 
problems  with  sufficient  accuracy,  its  designers  felt  it 
was  necessary  to  use  double-precision  arithmetic  through- 
out.  Both  these  features  slow  down  a  routine  (although 
tapes  are  not  used  for  very  small  problems) .   Use  of  the 
product  form  of  the  simplex  method,  as  in  this  routine, 
helps  greatly  in  reducing  the  computing  time  for  large 
problems,  but  interferes  considerably  with  flexibility. 

The  routine  RSMFOR  (MFOR  for  short)  uses  neither 
tapes  nor  double-precision  arithmetic,  but  operates  in 
the  product  form.   Speed  was  emphasized  in  its  design 
(although  some  time-consuming  work  is  done  for  the  sake 
of  problem  size),  and  its  flexibility  is  not  great. 

The  Bonner-Moore  routine  (BM  for  short)  operates  in 
the  standard  form  of  the  simplex  method  for  the  sake  of 
flexibility,  at  some  sacrifice  of  speed.   Use  of  tape 
allows  it  to  handle  rather  large  problems,  but  computing 
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time  probably  then  becomes  impractical ly  large. 

We  feel  that  each  of  these  routines  represents  just 
about  the  best  that  can  be  done  at  present  in  their  par- 
ticular directions  of  emphasis;  each  is  nearly  the 
optimal  solution  to  the  problem  of  designing  a  linear 
programming  routine,  under  the  constraints  of  current 
technology,  given  the  objective  of  maximizing  a  certain 
combination  of  each  of  the  measures  of  a  routine's  effec- 
tiveness—a combination  which  is  different  in  each  case. 
The  ordering  is  apparent  in  the  case  of  size.   As  far  as 
speed  is  concerned,  these  routine  were  recently  run  on 
most  of  the  linear  programming  test  problems  maintained 
by  the  SHARE  Linear  Programming  Project's  Committee  on 
Test  Problems  and  Experiments.   The  total  times  required 
to  solve  nine  of  these  problems  (having  from  20  to  117 
constraints)  by  each  of  the  routines  were:   LP90,  731  sec; 
MFOR,  338  sec;  BM,  590  sec. 

2 .   Input 

When  a  linear  programming  problem  is  stated  in  the 
form 
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(1) 


n 

Minimize         c.  x.   subject  to  x.  >  0  ,  (all   j)  , 
j=l   J   J  J 

m 
^  i=l  aiJ  XJ  =  bi  (±=l>">m)    ' 

as  is  the  case  with  almost  all  linear  programming  routines, 
only  the  quantities   c.  ,   a..  ,  and  b.   need  be  used 
as  input.   The  description  which  follows  incorporates  the 
standard  input  form  adopted  by  the  SHARE  organization  for 
linear  programming  routines,  and  is  that  used  by  most  of 
the  routines  mentioned  in  section  1. 

The  row  and  column  indicators  are  not  merely  taken 
as  numbers,  as  indicated  in  (1),  but  may  be  any  collec- 
tion of  six  alphanumeric  characters.   The  coefficients 
c.   are  treated  like  any  row  of  matrix  coefficients 
a..  (j=l,..,n)  ,  so  that  it  is  necessary  to  specify  which 

of  the  rows  of  data  read  in  holds  the  objective.   The 
right-hand  side  coefficients  may  also  be  treated  like 
any  column  a..  (i=l,..,m)   of  matrix  coefficients,  so 
that  it  will  be  necessary  to  specify  which  column  holds 
them.   All  these  coefficients  are  entered  on  cards--one 
per  card- -in  the  form  j  ,   i  ,   a. .  .   Thus  the  problem 
(2)  might  appear,  on  cards,  in  the  form  (3) .   The  first 
6  columns  of  the  card  are  reserved  for  needed  signals 
regarding  the  nature  of  the  data. 
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Minimize      $1.00    (Oats)    +  $0.50    (Hay)      subject    to 
(2)  65    (Oats)    +  10    (Hay)    >      51.5       (Calories) 

(Oats)    +  (Hay)    =      100       (Bulk) 


(3) 


MATRIX 

OATS 

MONEY 

1.00 

OATS 

HEAT 

65. 

OATS 

BULK 

1. 

HAY 

MONEY 

.50 

HAY 

HEAT 

10. 

HAY 

BULK 

1. 

SLACK 

HEAT 

-1. 

DIET 

HEAT 

51.5 

DIET 

BULK 

100. 

OBJECT 

MONEY 

RHS 

DIET 

In  the  last  few  years,  considerable  effort  has  gone 
into  the  writing  of  routines  which  would  simplify  the 
process  of  producing  input  in  this  standard  form.   Large 
linear  programming  models  are  often  built  up  in  a  formal 
way  from  data  not  presented  directly  as  activity  coef- 
ficients.  A  routine  which  performs  this  building  up  is 
called  a  "matrix  generator."  All  existing  matrix  gen- 
erators are  proprietary.   A  notable  general-purpose  gen- 
erator is  Standard  Oil  of  California's  "MATRAN,"  [2]  an 
augmentation  of  FORTRAN  which  permits  the  compilation  of 
elaborate  special-purpose  generators.   C-E-I-R  and  Bonner 
and  Moore  have  constructed  several  special-purpose  gen- 
erators for  particular  classes  of  oil  industry  problems. 
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Generators  are  of  great  value  when  many  problems  of  the 
same  kind  must  be  solved,  but  some  of  their  job  can  be 
accomplished  by  the  simpler  "file  maintenance"  features 
(means  for  ad  hoc  modification  of  input  data)  some  of 
the  standard  routines  possess. 

3.   Output 

The  standard  final  output  of  one  of  these  routines 
always  gives  the  activity  levels  of  the  basic  variables 
at  the  optimal  solution,  and  usually  the  reduced  costs 
for  all  the  activities.   Unless  it  is  a  standard  form 
routine  it  will  also  give  the  shadow  prices,  or  values 
of  the  dual  variables,  associated  with  all  the  constraints 
Arrangements  can  usually  be  made  for  obtaining  the  entire 
optimal  tableau.   It  is  generally  possible  to  obtain  all 
this  information,  or  a  short  summary  of  it,  for  each 
iteration  as  well.   An  important  feature  of  some  —  but  not 
all  —  routines  is  an  "error  check":   a  determination  of 
the  extent  to  which  the  final  primal  and  dual  solutions 
satisfy  their  constraints,  constituting  a  proof  of  the 
allegation  that  a  solution  has  been  obtained. 

In  large-scale  applications  of  linear  programming 
this  standard  output,  primarily  a  list  of  activity 
levels,  is  unsatisfactory  to  the  ordinary  user  of  the 
results  —  particularly  if,  through  the  use  of  a  matrix 
generator,  the  original  statement  of  the  problem  has 


t 


-8 


been  modified.   Consequently  "output  editors"  or  "report 
generators"  have  become  an  important  adjunct  in  appli- 
cations.  They  convert  the  standard  output  into  a  more 
easily  readable  form,  using  if  necessary  information 
about  the  prior  process  of  matrix  generation.   The  editor 
may  be  a  simple  routine,  quite  detached  from  the  linear 
programming  routine,  which  merely  processes  the  output  of 
the  latter;  with  IBM  machines,  the  1401  is  often  used  to 
edit  7090  output.   The  MATRAN  system  is  much  more  sophis- 
ticated:  After  generating  the  matrix,  it  retires  from 
the  computer,  returning  when  the  problem  has  been  solved 
to  generate  a  report  expressed  in  the  language  used  as 
input  to  the  generator. 

The  extensive  use  of  output  editors  —  all  of  which 
are  currently  proprietary—has  brought  about  a  substan- 
tial change  in  the  economics  of  the  use  of  linear  program- 
ming use  in  the  last  three  years.   It  is  not  uncommon 
that  the  majority  of  the  computer  time  spent  on  a  prob- 
lem is  devoted  to  generation  and  editing;  but  the  com- 
puter is  so  much  more  efficient  at  those  tasks  than  are 
people  that  the  net  cost  of  formulating  and  solving  prob- 
lems of  similar  types  has  decreased  a  great  deal  through 
these  aids. 
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4.   Computing  Methods 

The  only  computing  routines  we  know  of  in  successful 
use  for  linear  programming  employ  some  version  of  Dantzig's 
simplex  method,  although  other  procedures,  which  have 
more  general  problems  as  their  appropriate  domain,  can  be 
used  for  linear  programming.   Some  authors  have  described 
their  work  in  terms  which  obscure  its  connection  with 
the  simplex  method,  but  it  would  seem  historically  correct 
to  describe  any  procedure  which  pivots  in  a  tableau  as 
a  simplex  method. 

Variants  of  the  simplex  algorithm  arise  when  various 
rules  for  choice  of  pivot  row  and  column  are  employed; 
the  dual  simplex  method,  the  Dickson-Frederick  rule  for 
pivot  column  selection  [3],  and  the  "symmetric  algor- 
ithm" [4]  are  examples  which  have  been  promoted.   Inde- 
pendent of  the  algorithm  employed,  there  are  different 
forms  of  the  simplex  method--dif f erent  ways  in  which  the 
data  needed  for  pivot  selection  and  pivoting  are  calcu- 
lated.  Basically  these  are  the  standard  form,  in  which 
the  entire  tableau  is  maintained  and  updated,  and  the 
explicit  and  product  forms  (both  sometimes  called  the 
"revised"  simplex  method) ,  in  which  only  the  inverse  of 
the  current  basis  is  maintained.   There  is  some  interplay 
between  form  and  algorithm,  since  the  data  needed  for 
performing  a  certain  algorithm  may  be  more  conveniently 
obtained  in  a  certain  form. 
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Most  of  the  known  variants  of  the  simplex  method 
have  been  compared  in  their  behavior  on  the  test  prob- 
lems described  in  section  1.   The  results  of  this  exhaus- 
tive study  [5]  indicate  that  the  "ordinary"  simplex  algor- 
ithm (the  choice  of  pivot  column  by  minimal  reduced  cost, 
and  of  pivot  row  so  as  to  preserve  feasibility) ,  executed 
in  the  product  form,  and  augmented  by  certain  simple  de- 
vices ("suboptimization"  and  use  of  a  "full  basis"),  is 
the  most  efficient,  a  conclusion  which  seems  to  be  sup- 
ported by  what  comparisons  have  been  made  among  production 
routines . 

The  reasons  for  this  lie  more  in  the  efficiency  of 
the  product  form  than  in  that  of  the  ordinary  algorithm, 
since  many  of  the  algorithm  variants  (which  cannot  be 
done  well  in  the  product  form)  result  in  fewer  pivot  steps 
being  taken.   The  product  form  of  the  simplex  method  takes 
particularly  good  advantage  of  the  fact  that  all  large 
linear  programming  problems  are  sparse  —  that  is,  have 
relatively  few  nonzero  data.   The  relative  efficiencies 
of  the  standard,  explicit,  and  product  forms  on  the  test 
problems  were  1.0,  1.6,  and  2.5;  for  larger  problems 
these  numbers  become  farther  apart. 

Great  improvements  in  computing  efficiency  in  linear 
programming  have  resulted  from  an  improved  understanding 
of  the  computational  process.   Until  around  1958  it  was 
generally  believed  that  the  solution  of  linear  program- 
ming problems  having  more  than,  say,  100  constraints 
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would  always  require  double  precision  in  a  36-bit  ma- 
chine—that is,  considerably  more  than  9-digit  accuracy. 
Problems  having  around  400  constraints  are  now  solved 
routinely  with  9-digit  accuracy,  and  no  limits  of  this 
ability  are  known.   This  has  been  accomplished  largely 
by  giving  routines  the  ability  to  determine  when  they  are 
about  to  perform  a  calculation  likely  to  result  in  ex- 
cessive roundoff  error  (such  as  pivoting  on  a  very  small 
pivot  entry)  and  perform  some  other  calculation  instead; 
this  manner  of  operating  is  allowed  by  the  fact  that 
choices  available  in  performing  the  simplex  method  are 
often  not  uniquely  specified. 

A  nice  example  of  the  evolution  of  linear  program- 
ming routines  is  afforded  by  a  65-constraint  problem 
which  has  been  solved  many  times.   Using  the  best  avail- 
able routine  for  the  IBM  704  in  1957,  (RSLPS1)  40  minutes 
were  required  to  solve  it.   Several  IBM  7090  routines 
now  solve  it  in  less  than  50  seconds,  although  the  basic 
machine  speed  has  improved  by  no  more  than  a  factor  of 
six. 

5.   Decomposition  Routines 

There  has  been  considerable  interest  in  the  compu- 
tational implementation  of  the  decomposition  algorithm, 
but  so  far  most  decomposition  routines  have,  quite  prop- 
erly, been  constructed  for  experimenting  with  the 
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algorithm.   Only  the  first  of  those  listed  below  is 
available  as  a  production  routine. 

1.  SMDASS  and  SMDCOM,  respectively  the  data- 
assembly  and  computing  portions  of  the  Socony  Mobil  Oil 
Company  routine  for  the  IBM  7090  written  by  Arthur 
Friedenheit,  released  as  SHARE  Distribution  Numbers  1250 
and  1251  in  March,  1962.   This  routine  accommodates  25 
partitions,  and  the  extremal  program,  as  well  as  each 
subprogram,  may  not  exceed  90  constraints  or  500  vari- 
ables.  An  important  feature  is  the  special  handling  of 
subprograms  of  the  "transportation"  form,  each  of  which 
may  have  up  to  165  sources  and  4500  destinations.   This 
routine  has  been  thoroughly  checked  out,  but  its  lack  of 
flexibility  in  the  manner  in  which  the  decomposition  cycle 
is  undertaken  has  somewhat  limited  its  usefulness. 

2.  F.  Genuys  and  J.  M.  Gauthier  of  IBM  France  have 
done  a  few  experiments  with  a  small-scale  decomposition 
routine  for  the  IBM  704,  reported  in  "Experiences  sur  le 
Principe  de  Decomposition  des  Programmes  Lineaires," 
Premier  Congres  de  L 'Association  Francaise  de  Calcul, 
Gauthier-Villars,  Paris  1961,  373-382.   They  report  the 
solution  of  some  problems  with  decomposition  in  about 
one-eighth  the  time  required  by  the  standard  simplex 
method. 

3.  Pierre  Huard  and  others  of  Electricite  de  France, 
Paris,  are  currently  preparing  an  elaborate  large-scale 
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decomposition  routine  for  the  IBM  7090  aimed  at  both  ex- 
perimental work  and  production  use,  permitting  full  flexi' 
bility  in  performing  the  algorithm  by  different  routes 
and  even  the  solution  of  subproblems  by  decomposition. 
No  experience  has  been  reported  yet. 

4.  Bonner  and  Moore  have  completed  a  decomposition 
routine  for  the  IBM  7090  capable  of  handling  several 
thousand  partitions.   The  sizes  of  the  extremal  and  sub- 
problems  must  be  such  as  to  fit  into  about  20,000  words 
of  core  storage. 

5.  R.  Seaber  and  others  of  Shell  Oil  Company, 
Houston,  have  made  small  decomposition  routines  for  the 
IBM  650  and  7090,  reporting  mildly  encouraging  experi- 
ence . 

6.  Several  routines  have  been  written  embodying 
the  decomposition  principle  for  special-purpose  prob- 
lems:  One  by  Gomory  and  others  for  the  cutting-stock 
problem  (see  Chapter  VI,  Section  2) ;  by  Socony  Mobil  for 
the  transportation  problem  (Chapter  VI,  Section  5);  and 
it  appears  that  several  more  routines  of  this  kind  will 
be  completed  this  year. 
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IX-1 
DECOMPOSITION  BY  HAND 

Although  the  decomposition  procedure  is  intended  for  com- 
puters,   the  manual  solution  of  carefully  chosen  small  problems  can  be  a 
rewarding  experience  for  the  practitioner.     In  the  following,    we  summarize 
the  decomposition  algorithm  in  a  form  suitable  for  hand  computation,    and 
present  some  examples  for  solution.     It  will  be  assumed  in  the  instructions 
that  the  problems  are  small  enough  in  scale  that  the   set  of  extreme  points 
for  each  subproblem  can  be  enumerated  conveniently,    and  that  the  extremal 
problem  is  sufficiently  small  vertically  that  the   successive  bases  can 
readily  be  inverted  by  hand. 


A.     Notation 


Objective 
function 


X 


C 


X 


c 


Y 


Constant 


None 


Corporate 
constraints 


Subprogram 
1 


K, 


Subprogram 
2 


K 
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B.      Instructions 

1.  Insert  positive  and  negative  slacks  to  convert  all  inequalities  to 
equalities. 

2.  Compute  all  basic  feasible  solutions  for  each  subprogram. 

3.  Obtain  a  starting  solution  to  the  extremal  problem  -  defined  as  a 
basic  feasible  solution  involving  only  the  weights  and  perhaps  the 
corporate  (A)   slacks. 

3a.      If  all  the  original  constraints  have  positive  slacks,    a 

starting  solution  is     X.,  =  1  ;     u..  =  1  ;     (s,,  .  .  .  ,  s     )  =  (b,  .  .  .  ,  b     ) 

11  1  m  1  m 

-  where  the  extreme  points  X  and  Y  corresponding  to  \ 
and  (jl  involve  only  their  slacks  with  positive  weight.  The 
corresponding  starting  basis  is  an  identity  matrix. 

3b.      Trial  and  error.     If  there  are    k     positive  corporate 
slacks     s.   ,  s.    ,  .  .  .  ,    s.      ,      include  these  in  the  trial  starting 
solution  together  with    m+2-k     judiciously  chosen  weights 
(at  least  one  from  each  subprogram).      The  basis  will  be  made 
up  of  unit  vectors  for  the  slacks,    and  columns  of  the  type 
(AX     1  0)      or     (AY      0     1)      for  the  weights.      The  trial  solu- 
tion  is  the  coordinates  of  the  vector     B     (b    1    1)     ;    if  all  are 
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positive,    the  starting  solution  is  feasible.      If  not,    try  another 
set  of    m+2-k    weights. 

3c.      Modified  Phase  I  (see  notes).      This  method  is  rather 
tedious  for  hand  computation.     It  involves  introducing  a  com- 
plete set  of    m+Z    artificial  variables  into  the  extremal  problem, 
and  eventually  replacing  them  with  weights  and/or  corporate 
slacks.     As  an  alternative,    one  can  start  so  far  as  possible  as 
in  3a  -  i.  e.  ,    include  each  positive  corporate  slack,    and  a  single 
weight  of  value  unity  for  each  subprogram  with  all  positive  slacks 
and  add  an  artificial  variable  for  each  remaining  corporate  con- 
straint and  each  remaining  convexity  equation.      By  trial  and 
error  or  other  means,    one  would  then  attempt  to  replace  these 
artificial  variables  with  weights  and/or  corporate   slacks. 

4.      Compute  the  simplex  multipliers     (II    •     •  II     n-,n2^     from  this  formula 

(n  •  •  n    n.iL)  =  -  (c.  ,c  , . . .  ,c       )b~ 

1  m    1    2  i       i  l 

1        2  m+Z 

where    B     is  the  current  basis  and    c.   ,  .  .  .  ,  c.  are  the  coefficients 

l,  l 

1  m+2 

of  the  basic  variables  in  the  objective  function.     If  the  basic  variable 

0       is  an  artificial  or  slack  variable,    its     c.       is  zero;    if  it  is  a 

k  Xk 

weight    X    with  extreme  point    X       its    c.     is    C    X      ;    if  it  is  a  weight 

\  X 
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.  ,  *  # 

[i    with  extreme  point    Y      ,    its     o  is     C     Y 

'k--    Y       , 

4a.      Check  step.      (II    •  •  ■  n     n  n    )(b    1     1)  negative  of 

current    a     . 


5.      Compute  new  price  vectors      y      and     y       from  the  formulas 

v      =    C      +  (n    •  •  •  n     )A 
1  XI  ml 


\     =     CY  +  <V"nm>A2 

6.  Evaluate     v  X    for  each  extreme  point    X       of  the  first  subprogram, 
and  determine  which    X       minimizes     y  X     .      Compute 

A    =     IT     +  (min  y  X)   .     If    A    =  0  ,      go  to  step  7.     If    A    <  0,      go  to 
step  9. 

7.  To  be  done  only  if    A    -  0   .      Evaluate     y    Y     for  each  extreme  point 

Y       of  the  second  subprogram,    and  determine  which    Y       minimizes 

r 

v   Y   .      Compute     A     =  IT     +  (mm  v   Y)   .      If    A     =  0   ,      go  to  step  8.      If 

L*  &  £*  C*  Cd 

A     <  0,    go  to  step  9  • 

8.  To  be  done  only  if    A    =  A     =  0   .      This  means  that  current  solution 
is  optimal.      To  transform  back  to  the  original  variables,    use 

X  =    2  VXX*       ;    Y    =    S  M....Y* 

(i)     (i)  (i)    i 

where  the     \,  ,     and    ul,  ,     are  in  the  current  basic  set.      The  values 
(i)  (i) 

of  the  corporate  slacks  are  just  their  actual  values  (without  change) 
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in  the  current  basic   solution. 


9-     Suppose     A    <  0  ,    where     X        minimizes     y  X  .      Compute  ne 
i-  K  1 


w 


Identify  this  column  with    X      ,     the  new  entering  variable, 
10.    Compute  the    m+2    numbers 


1  2  m+2 


R,P  R,P  R       0P 

1    r  2     r  m+2     r 

where     9  .     is  the  value  of  the     i         basic  variable  and    R.     is  the     i 
i  i 

row  of  the  basis  inverse   -  all  from  the  previous  iteration.      Of  the 

above  quotients  which  are  positive,    pick  the  smallest     9     /(R    P    )   . 

s  s     r 

Then     9       is  the  new  departing  variable. 

11.    Replace     9        in  the  basic   set  by    X     ,     and  replace  its  column    P 
s  r  s 

in  the  basis  by    P      .     Invert  the  new  basis.      Compute  the  new  solution 
r 

and  the  new  objective  function  value  by 

"i     ■ 

=     B 
9  m+2 
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*     =     S  Ni)  1CX<)1  +     ^„,  [CYY,:i,   ] 

where     X         and    u.         are  in  the  current  basic  set. 
(i)  (i) 

11a.      Check  step.      Compute  the  difference     (New  a   -  Old  2)    ; 

and  compute  the  product    X     A,     ,      where     X       is  the  cum 

r      1  r 

value  of  the  just-entered    X      ,    and    A      is  the  negative  evaluation 
of  step  6.      These  two  results  should  be  equal. 

12.      Return  to  Step  4. 


C.     An  Example 

Minimize  the  linear  function     -Zx  -  5y  ,      subject  to  the  corporate 
constraint 

x  +  y  <      6 
as  well  as  to  the  divisional  constraints 

x  <    4 
and 

y  <    3 

Work  by  decomposition,    with  two  subprograms. 
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The  Problem    (with  slacks  s,t,v) 


x  y  s  t  v  b 

Min  a  -2  -5 

A 


A. 


1 

1 

1 

1 

1 

1 

1 

Cx  =     (-2     0) 
CY    =     (-5    0) 


Al     =     (  1    0) 
A2     =  (1    0) 


Basic  Solutions  of  Subprograms 


X 


X, 


X. 


x 

t 


Y 


y 

V 
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Starting  Solution 
All  Slacks  positive. 


Solution:    X.    =  1  ;     |j,    =  1  ;     s  =  6 


Basis : 


s 

h 

h 

1 

0 

0 

0 

1 

0 

0 

0 

1 

B     =  0  1  0  B 


a  =  0 


(n  n1  n2)   =   (o   o   o) 

v  X    =     (-2     0)X    is  minimized  by    X       with     Y  X     =     -8 
A    =     -8  .     Introduce     \       . 


First  Iteration 

Change  of  Basis:  s  X.  (jl  X 

10  0  4 

0  10  1 

0  0  1  0 

leave  enter 


s 

k2 

h 

1 

4 

0 

0 

1 

0 

0 

0 

1 

b" 
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Solution:     s  =  2   ;     \,     =  1  ;     u    =  1 

a  =  -a 


(n  n  n2)  =  (  o     8    o) 


"Y  X  =  (-2     0)X    is  minimized  by    X       with     "Y-,X 


=  -8 


A    =  0  .     Introduce  no  \  . 


v  Y  =  (-5     0)Y     is  minimized  by    Y       with     Y?Y 


-15 


A     =   -15  .     Introduce     |X     . 


Second  Iteration 


Change  of  Basis 


1 

4 

0 

1 

0 

0 

leave 

3 
0 

1 


enter 


B  = 


^2 

h 

^2 

4 

0 

3\ 

1 

0 

0 

0 

1 

1    , 

B 


-1 


0 

1 

1 

4 

3 

3 

,1 

4 

3 

3 

Solution:      X.     =  1  ;     \x 


1"   —'     ^2 


a  =     -18 
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(n  ni  n2)   =    (5      -12      0) 


Y.X  =     (3       0)  X     is  minimized  by     X      with     vX'     =     0 


A    =     -12  .      Introduce     \     . 


Third  Iteration 


Change  of  Basis: 


4 

h 

^2 

x, 

4 

0 

3 

0 

1 

0 

0 

1 

0 

1 

1 

0 

h 

\ 

^2 

4 

0 

3N 

1 

1 

0 

0 

0 

1 

leave  enter 


0  0 


4       \ 


'    '  T      l       T 


*  / 


3  1 

Solution:    \     -   —  ;    \    -  —    ;        |a     =    1 

z  =  -21 


(n  ni  n2)    =   (    2     0     9) 

v  X  =  (0    0)X    has  minimum  value     0   . 

A    =  0  .     Introduce  no  \  . 

v   Y  =     (-3      0)Y     is  minimized  by    Y        with     Y?Y       =     -9 

A     =  0  .      Introduce  no    (jl  . 


Current  solution  is  optimal. 
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Transformation  to  Original  Variables 

/  x)  1         #         3        *  /  3 

X=    (J     -    T    Xl    +   T*2  (l 


>:  -  <-(l. 


s     =     0 

The  optimal  solution  is  therefore 


X 

— 

5 

y 

= 

3 

s 

= 

0 

t 

= 

1 

V 

- 

0 

D.      Problem  for  Solution 


Minimize  the  linear  function 


B  =  -xx  -  x2  -  2yx  -    y2 

subject  to  the  corporate  constraint 

x    +  2x2  +  2y    +  y2  <    40 

and  to  the  two  sets  of  divisional  constraints 


x    +  3x     <      30 

1  "     — 


2x    +      x     <       20 

1  £*    — 


and 


IX-12 


<  10 

<  10 


Y^YZ    <     15 


Ans. 


=     -36 
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E.     Solution  to  Problem 


The  Problem  (with  Slacks     s ,  t  ,  t    ,  v  ,  v    ,  v   ) 


Xl        X2 


y, 


y2      s      tl 


Min  a 

A 


-1         -1 


2       -1 


2         1 


40 


30 

20 


A 


1 

1 

1 

1 

1 

1 

1 

10 
10 
15 


C__  =-(-1    -1    0    0) 

Cy  -  (-2     -10    0    0) 


A      =     (  1      2      0      0  ) 
A2    =     (  2      1      0      0      0) 


Basic  Solutions  of  Subprograms 
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'°\ 


10 
10 

i  5 


Y3 

I  10 

0 

0 

10 

5 


10 

5 
0 
5 
0 


/     5    \    ,'10 


0 

j\J 


Starting  Solution 
All  slacks  positive. 


Solution:    X    =  1  ;    \i    =  1  ;     s  =  40 


B  = 


1 

0 

0 

0 

1 

0 

0 

0 

1 

B 


-1 


/' 

0 

0 

0 

1 

0 

\o 

0 

1 

z  =  0 


(n  ni  n2)  =(ooo) 


v  X  =  (-1       -1      0       0)X     is  minimized  by     X       with     vX"    =    -14 


A,  =   -14  .     Introduce     X  A 
1  4 


First  Iteration 


IX  -  15 


Change  of  Basis 


s 

h 

*1 

\ 

1 

0 

0 

22 

0 

1 

0 

1 

0 

0 

1 

0 

leave 

enter 

B  = 


s 

h 

\ 

1 

0 

22 

0 

0 

1 

0 

1 

0  i 

-1 


B 


Solution:     s  =  18;     u,,  =  1  :      \     =1 

1  4 


z  =   -14 


(n  nx  n2)  =   (o     14    o) 

Y  X  =  (-1      -1      0      0)X    has  minimum     -14  .       A    =     0 


Y   Y  =  (-2      -1     0      0      0)Y     is  minimized  by    Y       with    min  = 


25 


A.   =   -25  .      Introduce     u.  .   . 
2  r4 


Second  Iteration 


Change  of  Basis 


s        ^ 

\ 

^4 

1        0 

22 

25 

0         0 

1 

0 

0         1 

0 

1 

leave 

enter 
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B  = 


^4 

h 

s 

lib 

0 

22* 

0 

0 

1 

\1 

1 

0 

-1 


Solution:     \     =  1  ;    jjl    = 


1 

25 

22 
25 

1 
25 

22 

25 

0 

1 

18 

1       25     ' 


25 


32 


(II  Hj  n2)  =  (1      -8      0) 

\  X  =  (010      0)X    is  minimized  by    X      or    X     ,    with 

min  =  0 


A    =   -8  .      Introduce     \      . 


Third  Iteration 


Change  of  Basis 


^ 


V- 


\ 


\ 


4        ^1  4  3 

25  0         22         10 

0  0  11 

110  0 

leave  enter 


B  = 


/25 
0 
1 


\ 


\ 

22 
1 
0 


Solution:     \x     =  I 


S  = 


1_ 
12 


B 


k4  ~    12 


0 
12 
12 
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z  =   -36  — 


1  20         50 


in  iij  n2>  =  , , 

vxx=   (-f  -t     ° 


V  =  (  "T 


0 


20 


0)X    has  min.    value     — —  .      A    =  0 

50 

0         0)Y     lias  min.    value     — —    . 


A2  =  0. 


Current  solution  optimal. 


In  terms  of  original  variables,    optimal  solution  is 


25 

1  "       3 

10 


x. 


H=  5 


\- 

35 
3 

t2  = 

0 

Vl  = 

0 

vz  = 

5 

V3  = 

0 
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DICTIONARY  OF  LINEAR  PROGRAMMING 
TERMS 


Accounting  Prices  .       Internal  prices  assigned  to  company -owned 
resources  as  an  aid  to  decision-making  and  control.      In  certain  linear 
programming  applications,    the  dual  variables  at  optimality  may  be  re- 
garded as  including  a  set  of  accounting  prices  for  the  various  scarce 
resources.     Accounting  prices  in  the  linear-programming  format  are 
called  shadow  prices  (q.  v.  ). 

Algorithm.     A   set  of  instructions  whose  rote  application  yields  the  solu- 
tion to  a  particular  class,  of  problem. 

Alternate  Solutions.      Distinct  optimal  solutions  to  the  same  optimization 
problem. 


Angular  Matrix.       A  block-triangular  matrix  (q.  v.  )  in  which  all  blocks 
are  zero  except  those  down  the  diagonal  and  along  the  bottom  row.      That 
is,    it  is  a  matrix  of  the  form 

A. 


11 


A 


22 


A.        A     r 

ml      mZ 


mm 


where  the    A        are  submatrices  (or  "blocks")  and  all  omitted  entries 
are  matrices  consisting  entirely  of  zeroes.      Linear  programming  pro- 
blems with  angular  matrices  lend  themselves  to  special  types  of  comput- 
ational treatment,    and  in  particular  to  that  of  decomposition. 


x-z 


Artificial  Variables.      Auxiliary  variables  introdu<  ed  LntO  the  equality 
constraints,   at  the  start  of  the  simplex  method  of  linear  programming, 
in  order  to  provide  an  identity  basis  so  that  the  optimization  procedure 
can  begin.      There  are  as  many  artificial  variables  as  there  are  equatii 
each  occurs  only  in  its  corresponding  equation,    with  the  coefficient     +1  . 
It  is  not  necessary  to  introduce  an  artificial  variable  in  a  converted  in- 
equality in  which  the  coefficient  of  the  slack  variable  is     +1  ,      since  the 
necessary  unit  vector  is  already  present  for  this  equation.      The  artificial 
variables  (and  positive  slacks)  constitute  an  initial  basic  feasible  solu- 
tion,   but  to  a  changed  problem;     once  the  artificial  variables  have  been 
driven  out  of  the  basic   set,    by  simplex  pivot  steps,    the  resulting  basic 
variables  constitute  an  initial  basic  feasible  solution  to  the  original 
problem. 

Artificial  Vector.       The  column  vector  associated  with  an  artificial  vari- 
able.    It  is  a  unit  vector,    with    +1    in  the  row  where  the  artificial  variable 
occurs,    and  zero  elsewhere. 

Basic  Solution.     A  basic  solution,    in  linear  programming,    is  a  solution 
which  is  an  extreme  point  or  vertex  of  the  convex  solution  polyhedron 
determined  by  the  constraints.      In  a  problem  with    m    constraints,    a 
basic   solution  cannot  have  more  than    m    of  its  coordinates  different 
from  zero.      If  exactly    m    coordinates  are  non-zero,    the  basic   solution 
is  termed  non-degenerate;    if  fewer  are  non-zero,    it  is  called  degenerate. 
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Basic  Variables.        Those  coordinates  which  can  be  non-zero  at  an 
iterative  stage  of  the  solution  of  a  linear  programming  problem.      The 
basic  variables  carry  all  the  physical  weight  of  the  solution;     and  their 
column  vectors  together  comprise  the  current  basis  matrix. 

Basis.      Any  maximal  set  of  linearly  independent  vectors  in  a  vector 
space.      Each  vector  in  the  vector  space  is  expressible  as  a  unique  com- 
bination of  the  vectors  of  a  given  basis.     All  bases  of  a  given  vector  space 
contain  the   same  number  of  vectors;    this  common  number  is  called  the 
dimension  of  the   space. 

In  linear  programming,    the  unqualified  term  "basis"  invariably  is 
that  of  the  vector  space  spanned  by  the  column  vectors  of  the  coefficient 
matrix  of  the  constraints.      With    m    independent  constraints,    this  vector 
space  has  dimension    m    and  the  chosen  basis  usually  consists  of    m 
independent  columns.      Sometimes  the  word  "basis"  is  used  loosely  to 
denote  the   square  matrix  formed  by  the  display  of  such  a  basis. 

Basis  Inverse.       The  inverse  of  the  basis  matrix. 

Basis  Matrix.        The   square  matrix  formed  by  the  display  of  the  basis 
vectors. 

Basis  Vectors.      The  individual  vectors  comprising  the  given  basis  of  a 
vector  space.      The  basis  vectors  in  a  linear  programming  iteration  are 
the  columns  corresponding  to  the  non-zero  variables  in  the  current  solu- 
tion.     They  form  a  basis  of  the  vector   space  spanned  by  the  coefficient 
columns. 


X-4 


Block-Triangular  Matrix.        A  generalization  of  the  notion  of  triangular 
matrix  (q.  v.).     A  block-triangular  matrix  is  one  that  can  be  written 
in  the  form 

"  A. 


A 


11 

21 


A 
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where  the    A.,     are   submatrices  (or  "blocks")     and  all  entries  not  shown 
are  submatrices  consisting  entirely  of  zeroes.      The    A.,     need  not  be   square; 
but  the  vertical  dimensions  of  all  blocks  in  one  row  must  coincide,    as  well 
as  the  horizontal  dimensions  of  all  blocks  in  one  column.      Practical 
linear -programming  problems  frequently  have  block-triangular  matrices; 
when  this  is  the  case,    the  computational  work  of  obtaining  a  solution  can 
often  be   shortened  considerably. 

Consistent  Equations.       Equations  which  can  be  satisfied  simultaneously; 
equations  for  which  a  set  of  values  of  the  variables  exists  which  satisfies 
all  the  equations.     A   set  of  linear  equations  is  consistent  if  and  only  if 
the  rank  of  its  matrix  of  coefficients  equals  the  rank  of  the  matrix  obtained 
by  augmenting  the  coefficient  matrix  with  the  column  of  constant  terms. 


Constraint.       An  equation  or  inequality  restricting  the  independent 
variables  in  an  optimization  problem.      Equations  directly  or  indirectly 
involving  the  yield  variables,    such  as  optimality  conditions,    are  not 
usually  considered  to  be  constraints.      The   set  of  all  constraints  taken 
together  determines  the  feasible  range  of  variation  of  the  independent 
variables . 
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Constraint  Matrix.     The  term  has  meaning  only  in  problems  where  the 
constraints  are  all  linear  equations,    as  in  linear  programming.      In 
such  cases,    the  constraint  matrix  is  the  matrix  formed  by  the  columns 
of  coefficients  and  the  column  of  constants. 

Convex  Combination.        The  convex  combination  of    r     points     X  ,    X    ,  .  .  .  ,  X 

in  a  Euclidean  space  of  arbitrary  dimensionality  is  a  particular   sort  of 

linear  combination  of  these  points,    and  is  thus  another  point     X    in  the 

same   space.      Specifically,      X    is  a  convex  combination  of    X  ,  X    ,  .  .  .  ,  X 

1       Z  r 

provided 

r 
X    =      2     X..X. 
i=l       X    X 

where  the     \.     are  non-negative    numbers   summing  to  unity.      The  set  of 

all  convex  combinations  of    X,,  X_,  .  .  .  ,  X       is  called  a  convex  set. 

1       Z  r 

A  convex  combination  of  two  points  lies     on  the  line   segment  joining 
them;    a  convex  combination  of  three  points  lies  in  the  triangle  formed 
by  the  three  points,  ;     of  four  points,    in  their  determined  tetrahedron,    and 
so  on. 

Convex  Function.       The  function    f(x)     is  convex  on  the  interval    (a,  b) 


provided 


f(\Xl  +  jix2)  <     \f(Xl)   +  (J.f(x2) 


for     a  <  x  ,  x    <  b     ;     0  <  \   ,      |j.  <  1  ;     \.  +  jx  =  1  .     An  equivalent  geometric 
statement  is  that  all  chords  of  the  curve     y  =  f(x)     lie  above  the  corres- 
ponding arcs  on  (a,  b)   .      Convex  twice  differentiable  functions  are  those 
for  which  the  second  derivative  is  nowhere  negative.      If    f(x)     is  convex  on 
(a,b)   ,      its  maximum  on  this  range  must  occur  at  one  of  the  end  points 
a    or     b  . 
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Convex  Polyhedron.      A  particular  type  of  convex  set  encountered  fre- 
quently in  linear  programming  theory;    defined  as  the   solution  set  of 
a  finite  system  of  linear  inequalities     (  <  )   •      If  it  is  a  bounded  set  then 
it  is  also  the  convex  set  made  up  of  all  convex  combinations  of  a  finite 
number  of  extreme  points  (vertices). 

Convex  Programming.      Minimization,    subject  to  linear  inequality  con- 
straints on  the  variables,    of  objective  functions     z    of  the  form 

z  =  f  (x  )  +  f   (x   )  +  ■  •  •    +  f  (x  ) 
1     1  Z      Z  n     n 

where  each    f.(x.)     is  a  convex  function.      Special  cases  are  linear  pro- 
11  r  r 

gramming,    where  each    f.(x.)     is  of  the  form    c.x.   ,      and  quadratic  pro- 

11  11 

gramming,    where  each  is  of  the  form    c.x. 

Convex  Set.         A  point  set,    in  Euclidean  space  of  any  dimensionality, 
with  the  property  that  the  convex  combination  of  any  collection  of  points 
in  the  set  is  also  in  the   set.     An  equivalent  and  simpler  definition  requires 
that  the  convex  combination  of  any  two  points  in  the  set  remains  in  the 
set.      Most  of  the  points  in  a  convex  set  can  be  expressed  as  a  convex 
combination  of  other  points  in  the  set;    the  relatively  few  points  which 
cannot  be  so  expressed  are  called  extreme  points  or  vertices. 

Decomposition.       A   scheme  for  adapting  certain  types  of  linear  programs 
with  very  many  equations  to  computers  which  lack  the  capacity  to  solve 
such  systems  in  the  conventional  way.      Linear  programs  which  qualify 
for  decomposition  are  those  for  which  the  variables  can  be  separated 
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into  classes     X,    Y,    Z,  .  .  .      and  the  constraints  into  classes   ft    ,  ft  ,  ft    ,    .  .  . 
such  that:     (a)     the  constraints  in  class    ft       involve  all  the  variabli 
(b)     the  constraints  in  class    ft      involve  only  the  variables     X   ;     (i  ) 
constraints  in  class    ft       involve  only  the  variables     Y    ;     (d)     the  CO  Lnts 

in  class    ft       involve  only  the  variables     Z   ;    and  so  on.      The   subsets  of 
constraints  each  involving  only  a  part  of  the  variables  are  called  subpro- 
grams. 

Linear  programming  by  decomposition,    just  as  with  ordinary  methods, 
involves  finding  successively  better  solutions  of  the  total  program.      The 
difference  lies  in  the  way  these  improved  solutions  are  obtained.      In 
ordinary  linear  programming,    one   stays  with  the  given  vertical  dimen- 
sionality,   and  passes  from  one  extreme  point  to  the  next  by  finding  a 
column  vector  with  a  negative  evaluation.     In  decomposition,    one  first 
reduces  the  vertical  dimensionality  by  transforming  to  an  equivalent 
problem  with  more  variables  and  fewer  restrictions.      The  resulting  "long- 
and-narrow"  program  is  in  theory  suitable  for  the  revised  simplex 
method  of  solution;    but,    since  the  transformation  is  conceptual  rather 
than  actual,    the   set  of  new  columns  is  never  available  in  its  entirety; 
and  some  substitute  expedient  is  required  to  replace  the  conventional 
list  of  evaluations.     In  decomposition,    the  transition  from  one  extreme 
point  to  the  next  is  accomplished  by  finding  optimal  solutions  of  certain 
of  the  original  subprograms,    each  relative  to  an  auxiliary  objective 
function  whose  coefficients  vary  with  the  iterative  stage.      The  algorithm 
also  includes  an  optimality  test  at  each  stage,    and  a  means  of  transforming 
an  optimal  solution  in  the  new  variables  back  to  the  original  variables. 
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Degenerate  Solution.      In  linear  programming,    a  degenerate  solution 
a  basic   solution  in  which  the  number  of  non-zero  variables  is   smaller 
than  the  number  of  independent  constraints. 

Departing  Variable.       That  non-zero  variable,    at  an  iterative   stage   Ln 
the   solution  of  a  linear  programming  problem,    which  is  to  become  zero 
in  the  next  iteration.      In  ordinary  (primal)   simplex  methods,    the  choice 
of  departing  variable  is  determined  by  the  choice  of  entering  variable. 

Direct  Problem.        In  linear  programming,    the  problem  as  originally 
stated,    as  contrasted  with  its  dual. 

Directional  Derivative.        Of  a  function    u(x,  y,  z)     at  a  given  point  in  a 
given  direction  -  is  the  rate  of  change  of    u(per  unit  distance)     at  that 
point  in  that  direction.      Its  value  may  be  calculated  by  taking  the  component 
of  the  gradient  of    u    in  that  direction;     or  by  use  of  the  formula 


du  Qu  3u 

=    \  +  p.    — — 

ds  9x  Sy 


Ml 


+      V 


bz 


where     (\,  \i,v)     are  the  direction  cosines  which  specify  the  direction  s   . 
The  directional  derivative  takes  its  largest  value  at  a  point  in  the  direction 
of  the  gradient. 


Dual  Problem.        Let  the  original  or  direct  linear  programming  problem 

be  that  of  determining  non-negative  values  of    x  ,  x    ,  .  .  .  ,  x       so  that 
n  1      Z  n 

z     =         2     c  x.     is  minimized 

"    j=i  J  J 
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subject  to 


n 


2      a..x.  >      b 

J=l         J 


n 


2   a.  .x     =      b 


(  i  =  1,  2, k) 

(  i  =  k  +  1,  .  .  .  ,  m) 


j  =  l 


The  dual  problem  is  defined  to  be  that  of  determining  values  of 

w  ,  w    ,  .  .  .  ,  w       ,    where     w.   •  •  ■  w,      are  to  be  non-negative  and    w 
12m  1km  k+1 

are  unrestricted  in  sign,    so  that     z  =     2     b.w.     is  maximized  subject  to 

-.,11 
m  1=1 


w 


m 


2    a.  .w.   <    c . 


i=l 


ij    i  =       J 


(  j  =  1,  2,  ..  .,n) 


A  useful  mnemonic  device  relating  the  direct  and  dual  problems  is 


Xl 

X2 



X 

n 

wl 

ail 

ai2 



ain 

bi 

wk 

\l 

ak2 



akn 

\ 

wk+l 

\+l, 

1 

ak+l,  2 



k+1,  n 

\+i 

w 

a     i 

a      -> 

a 

b 

m 

ml 

mZ 

mn 

m 

Cl 

C2 



c 

n 

I 

where  the  direct  problem  is  read  across  the  diagram  and  the  dual  is 
read  down. 
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Dual  Simplex  Algorithm.       One  of  the  several  algorithms  which  have  been 

devised  to  solve  the  linear  programming  problem.      The  algorithm  rests 

on  the  fact  that  the   set  of  extreme  points  with  all  evaluations  non-negative 

intersects  the  infeasible  region  in  a  chain  of  adjacent  vertices.      To  start 

the  algorithm,    one  requires  a  basic  solution  to  the  primal  problem  with 

all  evaluations  non-negative;     it  will  in  general,    however,    be  infeasible: 

i.  e.  ,    will  have  at  least  one  of  its  coordinates  negative.      The  departing 

variable     x       is  chosen  first,    and  is  taken  to  be  that  coordinate  which  is 
r 

most  negative.      Once     x       is  chosen  to  leave,    the  entering  variable     x 

r  s 

is  determined  by  taking     s     to  be  that    j     which  maximizes 


E    P    <  0 
r    J 


B    P 
r    J 


where     A.     is  the  evaluation  for  the  non-basic  variable     x     ;     P      is  its 
J  j  J 

column  in  the  constraint  matrix;    and    B       is  the  row  of  the  current 

r 

basis  inverse  corresponding  to    x     .      The  new  solution  will  again  have 

r 

all  evaluations  non-negative,    but  the  value  of  the  dual  objective  function 
will  have  been  increased.      The  process  terminates  when  the  latter  reaches 
its  maximum  value;    the  primal  solution  will  then  have  no  negative  co- 
ordinates,   and,    being  feasible,    will  be  optimal.      Except  for  the  rules 
for  choosing  the  departing  and  entering  variables  at  any  stage,    the 
computations  proceed  exactly  as  in  the  revised  simplex  method. 


Duality  Theorem.      If  either  the  direct  problem  or  the  dual  problem  has 

a  finite  optimum,    then  the  other  problem  also  has  a  finite  optimum  with 

the  same  optimal  objective  function  value  (i.  e.  ,      a  for  the  direct 

^  J  mm 

problem  equals     z  for  the  dual.  )     But  if  either  problem  has  an 

max 
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infinite  optimum,    then  the  other  problem  has  no  feasible  solutions. 

Dual  Variables.       The  optimal  values  of  the     m  +  n    variables  of  the  dual 

problem.      These  are  made  up  of  the     m     structural  variables  (shadow 

prices)     w  ,  w    ,  .  .  .  ,  w        along  with  the    n    dual  slacks  (final  evaluatio. 
I       £■  m 

s  ,  s    ,  .  .  .  ,  s      .     At  least    m    of  the  dual  slacks  are  zero,    correspond, 
to  the  optimal  basic  variables  of  the  primal  problem.      The  remaining 
n-m    dual  slacks  are  in  general  positive  (except  for  degeneracy)  and 
combine  with  the     m     shadow  prices  to  furnish  the  physical  solution  to 
the  dual  problem. 

Elementary  Operations.       Any  of  the  following  operations  on  the  rows  of 
a  matrix;    (i)     interchanging  two  rows     (ii)     multiplying  a  row  by  a  non-zero 
constant    (iii)     adding  a  multiple  of  a  row  to  some  other  row.      Elementary 
operations  do  not  affect  either  the  rank  of  a  matrix  or  the  solutions  of 
the  corresponding  set  of  linear  equations. 

Entering  Variable.         In  linear  programming,    that  non-basic  variable  of 
the  present  iteration  which  is  to  become  basic  in  the  next  iteration. 
Equivalently,    it  is  that  variable  which  is  zero  in  the  current  iteration  and 
which  is  to  become  non-zero  in  the  next.     Any  zero  variable  with  a  negative 
evaluation  may  act  as  the  entering  variable;     in  practice,    the  variable 
whose  evaluation  is  most  negative  is  often  the  one  selected. 


Equivalent  Equations.        Two  equations  in  the  same  unknowns  are  called 
equivalent  if  they  have  the   same  set  of  solutions.      Similarly,    two  systems 
of  equations  in  the  same  unknowns  are  equivalent  if  any  solution  of  either 
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system  also  satisfies  the  other  system.  .Solution  of  a  system  of  Linear 
equations  is  usually  accomplished  by  transforming  to  a  series  of  equi- 
valent and  progessively  simpler   systems. 

Evaluations  or  Evaluators.  Certain  numbers  associated  with  the  non- 
basic  variables  at  an  iterative  stage  in  the  solution  of  .1   Linear  programming 

problem.      Let  there  be     m    constraints;     let  the   simplex  multiplie  r  s 

(q.  v.  )  for  this  stage  be     X, ,  X_,  .  .  .  ,  X         ;    and  let  the  objective  function 

12m 

be     ex    +  c_,x_,  +  •  •  •    +  c    x       .      Then  the  evaluation    A       for  the  non-basic 
1    1  Z    Z  n    n  k 

variable     x,      is  defined  to  be 
k 

A     =    c,    +  (X    •  •  •    X     )P. 

k  k  1  m      k 

where     P       is  the  column  of  the  constraint  matrix  associated  with  the 
k 

variable     x     .      Numerically,    the  evaluation    A       is  the  change  in  the 

k  k 

objective  function  which  results  wnen  the  variable     x,      is  increased  from 

k 

0     to     1    and  the  basic  variables  are  shifted  in  value   so  that  the  constraints 
remain  satisfied.      If  all  of  the     n-m    evaluations  are  positive  or  zero, 
then  the  current  basic   solution  is  optimal  (yields  a  true  minimum.  ) 

Extreme  Point.  A  point  of  a  convex  set  which  cannot  be  expressed  as 
the  convex  combination  of  other  points  in  the  set.     Syn.  :  vertex. 


Feasible  Solution.       A   set  of  values  of  the  independent  variables  which 
satisfy  all  the  constraints;     as  opposed  to  an  optimal  solution,    which 
minimizes  the  value  of  the  objective  function  as  well.      In  linear  pro- 
gramming,   the  term  has  the  added  connotation  of  a  solution  to  the  con- 
straint equations  in  which  all  structural  variables  and  physical  slacks  are 
non-negative  and  all  artificial  variables  are  at  zero. 
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Gradient.       A  vector  function  which  charat  terizes  the  spatial   rate  of 

change  of  some  point  function  in  three-space.      The  gradient  of  the  point 
function      cf>(x,  y,  z)     denoted  by  grad     cj)    or    V<j>  .    is  defined  as 

grad  $    =     171-    +    J  .  +    k  7H1- 

&  Y  ox  J   by  by. 

where     i,    j,    and    k    are  unit  vectors  parallel  to  the     x-,    y-,    and     z-axes 
respectively.      The  directional  derivative  of    cf>  ,     at  any  given  point  in  any 
given  direction,    is  equal  to  the  dot  product  of  the  gradient  at  that  point 
and  the  unit  vector  in  that  direction.      Consequently,    the  direction  in 
which     cj)    changes  most  rapidly  is  the  direction  of  the  gradient  itself. 

Hyperplane.      An    (n-1)-   space  in  n-space;    the  set  of  all  points 

(x.,x_,  .  .  .  ,x   )     in  n-space  whose  coordinates   satisfy  some  linear  equation 

1      c  n  '  * 

2  a.x.   =  b  . 
i    i 

Identity  Matrix.        A   square  matrix  with  diagonal  elements  all  unity  and 
all  other  elements  zero.      The  identity  matrix    I    plays  the  same  role  in 
matrix  algebra  that  unity  does  in  the  algebra  of  real  numbers;     i.  e.  , 
AI  =  IA   =  A     for  any  conforming  square  matrix    A    ;    also,    it  is  the  fixed 
element  with  respect  to  which  inverses  are  defined. 


Inconsistent  Equations.       Equations  which  cannot  be   satisfied  simultane- 
ously;    equations  for  which  no  single   set  of  values  of  the  variables  exists 
which  satisfies  all  the  equations.     A  necessary  and  sufficient  condition 
for  a  system  of  linear  equations  to  be  inconsistent  is  that  the  rank  of  its 
coefficient  matrix  be  smaller  than  the  rank  of  its  entire  matrix  (i.  e.  , 
its  matrix  with  the  constant  column  included). 
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Independent  Equations.       A   set  of  equations  none  of  which  can  be  expret 
as  a  linear  combination  of  the  others.     A  necessary  and  sufficient  condition 
for  the  independence  of  a  set  of  linear  equations  is  that  its  matrix  (in- 
cluding the  constant  column)  have  rank  equal  to  the  number  of  equations. 
Thus,    a  system  of  linear  equations  whose  number  is  at  least  two  more 
than  the  number  of  variables  is  never  independent;     while  if  the  equations 
number  less  than  this,    they  may  or  may  not  be  independent. 

Infeasible  Solution.      An  impossible  construction,    except  in  linear  program- 
ming where  the  term  is  used  in  the  special  sense  of  a  solution  to  the  con- 
straint equations  in  which  either  some  intrinsically  non-negative  variables 
are  negative  or  some  artificial  variables  are  non-zero. 

Inverse,    of  a  Matrix.        Defined  only  for  a  square  matrix  with  a  non-zero 
determinant.     An  inverse  of  such  a  square  matrix    A     is  another   square 
matrix    B     of  the  same  dimensionality  such  that    AB  =  BA  =  I  ,    where     I 
is  the  identity  matrix.      It  follows  that  the  inverse  of  a  matrix,    if  it  exists 
at  all,    is  unique.      The  inverse  of    A     may  be  obtained  by  replacing  each 
element  of    A     by  its  cofactor,    and  then  dividing  each  element  of  the 
resulting  matrix  by  the  determinant  of    A   .      This  method  is  impractical, 
however,    if    A     is  at  all  large;     and  other  means  have  been  devised  which 
are  better  for  computation. 

Linear  Combination.       An  expression  of  the  form 


Vl  +  a2Z2  + 


+  a    z 
n   n 


where  the     a.     are  ordinary  numbers  and  the     z.     are   symbols,    such  as 
i  i 

vectors,    which  can  be  multiplied  by  numbers. 
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Linear  Inequality.        A   condition  of  the  form 


ax,  +  a_x_,  +  •  •  •    +  a    x       R  a„ 
112    2  n   n  0 

where  the     a.     are  constants,    the     x      are  variables,    and    R     is  any  one 

i  i  ' 

of  the  four  relations     >,>,<,      <     . 


Linear  Programming  Problem,    The.      Minimize  the  linear  function 


a     =     c,x,  +  c_x_  +  •  •  •    +  c    x 

11  2    2  n    n 

over  all  those  points  in  the  region 

x   >  0  ;    x,  >  0  ;  •  ■  ■    ;    x     >0 

1  —  2  —  n  — 

which  also  satisfy  the     m    linear  relations 


ailXl    +    ai2X2     + 


+  a     x       R     b 
In    n        11 


a21Xl    +    a22X2     + 


+  a     x      R     b 

Zn    n       2      2 


a     -x.    +   a      _x_  +  •  •  •    +  a        x     R     b 

ml    1  m2    2  mn    n      mm 

Here  each    R.     is  some  one  of  the  three  relations     >  ,    <  ,    =     .    Geom- 
etrically,   an  inequality  constraint  of  the     >    or     <    type  requires  that 

the  point    (x.,  x_,  .  .  .  ,  x   )     lie  on  a  certain  closed  half-space  in    E      , 
1      2  n  n 

while  an  equality  constraint  requires  that  it  lie  on  a  certain  hyperplane. 
An  exact  solution  to  this  problem  is  furnished  by  the  simplex  method 
(q.  v.  )     and  its  variants. 
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Marginal  Cost.       The  first  difference  of  cost  as  a  function  of  quantity; 
or,    in  the  continuous  case,    the  first  derivative.      If    x    denotes  the  number 
of  units  bought  or  produced,    and    C(x)     denotes    the  cost  of  these     x    unit 
the  marginal  cost    c(x)     is  a  second  function  of    x    defined  formally  by: 


c(x)     = 


C(x+1)    -  C(x) 
dC/dx 


(x  discrete) 
(x  continuous) 


Marginal  Revenue.  The  first  difference  (or  derivative)  of  revenue  as  a 
function  of  the  quantity  sold.  If  R(x)  denotes  the  total  income  from  the 
sale  of  x  units,  the  marginal  revenue  is  another  function  r(x)  defined 
formally  by: 


r(x) 


R(x+1)   -  R(x) 
dR/dx 


(x  discrete) 
(x  continuous) 


Non-Basic  Variables.       Those  coordinates  at  a  linear  programming 
iterative  stage  which  are  fixed  at  zero,    and  whose  column  vectors  do 
not,    therefore,    appear  in  the  current  basis.      In  contrast,    the  basic 
variables  carry  positive  weight  and  their  column  vectors  appear  in  the 
basis. 

Non-degenerate  Solution.       In  linear  programming,    a  non-degenerate 
solution  is  a  basic  solution  in  which  the  number  of  variables  with  non- 
zero values  is  equal  to  the  number  of  independent  constraints. 


Non-singular  Matrix.      A   square  matrix  whose  determinant  is  not  zero, 
and  thus  whose  rank  is  equal  to  its  dimension.     Any  non- singular  matrix 
has  an  inverse. 
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Objective  Function.        That  function  of  the  controllable  variables  which 
is  to  be  maximized  or  minimized  in  an  optimization  problem. 

Opportunity  Loss.       The  cost,    in  an  optimization  problem,    of  choosinj 
a  non-optimal  course  of  action;     also  called  regret.      Let     Q    denote  the 
set  of  all  relevant  courses  of  action  G  ;    let    f(0  )     be  the  objective  function; 
and  let    f       be  the  optimal  (maximum  or  minimum,    as  the  case  may  be) 
value  of    f(0)     for     0e  Q    .       Then  the  opportunity  loss     L(  0)     associated 
with  the  course  of  action     0     is  defined  as 

L(0)  -     |   f(0)   -  f*    |     . 

Optimal  courses  of  action  are  then  characterized  by  an  opportunity  loss 
of  zero. 

Pivot  Element.       That  element  of  the  matrix  of  a  set  of  linear  equations, 
at  a  step  in  a  pivotal  method  of  solution,    which  appears  in  the  recom- 
putation  formulae  for  all  elements  of  the  matrix.     In  the  particular 
pivotal  process  of  reducing  all  but  one  of  the  elements  of  a  given  column 
to  zero,    by  adding  multiples  of  a  fixed  row  to  the  other  rows,    the  pivot 
element  is  the  unreduced  element  of  the  column. 


Pivot  Step.      A   step  consisting  of  a  single  transformation  of  the  matrix 
in  a  pivotal  method  for  the  solution  or  reduction  of  a  system  of  simul- 
taneous linear  equations. 
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Pivotal  Method.        One  of  the  many  methods  used  in  the  solution  of  :jets 
of  linear  equations,    in  which  some  particular  coefficient  plays  a  dom- 
inant role  at  each  stage  of  the  elimination  or  condensation  process.      One 
such  method,    which  forms  the  base  of  the  ordinary  simplex  method  of 
linear  programming,    is  called  the  method  of  multiplication  and  subtraction. 
A   step  in  this  method  involves  transforming  the  equations  so  that  a  desi- 
gnated variable     x       shall  appear  only  in  the     r         equation  and  no  others. 


To  accomplish  this,    row    r     of  the  matrix  is  first  divided  by    a 


r  s 


and  suitable  multiples  of  this  row  are   subtracted  from  all  other  rows   so 
that  the  coefficient  of    x       in  the  other  equations  is  zero.      The  resulting 


transformation 

1  £ 

f 

1 

a. .    = 

a   . 
ij 

a 

r  s 

1 
a, .    = 

a 

is 


rs 


(  i  =  r) 


rj 


(i  \  r) 


Since  the  coefficient    a         appears  in  the  computation  of  each  new  co- 
efficient,   it  is  called  the  pivot  element.      Variants  of  the  simplex  procedure, 
such  as  the  revised  simplex  method,    involve  other  types  of  pivotal  methods. 

Primal  Problem.       Same  as  direct  problem.      The  linear  programming 
problem  as  originally  stated;  the  dual  of  the  dual. 


Primal-Dual  Algorithm.        One  of  the  more  recent  linear  programming 
algorithms,    which  attempts  to  combine  the  best  features  of  the  direct  and 
dual  methods.      In  the  ordinary  two-phase  method  of  linear  programming, 
the  first  phase  is  concerned  with  finding  a  basic  feasible   starting  solution, 
and  does  this  by  introducing  artificial  variables  and  driving  them  all  to 
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zero.      The  basic  feasible   solution  so  obtained,    however,    may  be  far 
from  optimal.      In  theprimal-dual  algorithm,    the  sequence  of  computations 
is  such  that  when  the  artificial  variables  are  finally  driven  to  zero,    the 
resulting  basic  solution  is  not  only  feasible  but  optimal.      The  process 
starts  with  a  solution  to  the  dual  problem.      Corresponding  to  this  solution, 
a  restricted  primal  problem  is  written  down  and  solved  in  which  some  but 
not  all  of  the  primal  variables  are  allowed  to  enter  the  basic   set       Then 
a  new  solution  to  the  dual  problem  is  found,    with  a  larger  value  of  the 
objective  function;    a  new  restricted  primal  problem  is  written  down  and 
solved;     and  so  on.      Eventually  a  solution  to  the  dual  is  obtained  which 
maximizes  its  objective  function.      The  corresponding  primal  solution  will 
be  free  of  artificial  variables,    and  will  constitute  an  optimal  solution  to 
the  original  linear  programming  problem. 

Rank.        The  maximum  number  of  linearly  independent  rows  of  a  matrix; 
equivalently,    the  maximum  number  of  linearly  independent  columns,    or 
the  size  of  the  largest  square  submatrix  with  a  non-zero  determinant. 


Redundant  Equations.        Equations  which  are  at  once  consistent  and 
dependent.      The  justification  for  the  word  "redundant"  is  as  follows.      The 
information  conveyed  by  a  system  of  equations  in    n    unknowns  may   be 
regarded  as  narrowing  the  entire    n-space  down  to  a  subspace  of  solutions. 
If  one  of  the  equations  can  be  expressed  as  a  linear  combination  of  the 
others,    its  omission  from  the  system  does  not  increase  the  subspace  of 
solutions  and  thereby  decrease  information. 


i 
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Revised  Simplex  Method.       A  variant  of  the  simplex  method,    suitable  for 
"long-and-narrow"  linear  programs  where  the  number  of  constraints     n 
is  much  larger  than  the  number  of  constraints     m  .      The  revised  method 
employs  a  fixed  set  of  equations  and  a  shifting  basis,    as  compared  with 
the  ordinary  method  which  uses  a  shifting  set  of  equations  and  a  constant 
(identity)  basis. 

The  basis     B  ,     which  is  a  particular  selection  of    m    columns  from  the 
coefficient  matrix  of  the  constraints,    serves  to  identify  or  mark  the  non- 
zero variables    X    at  any  stage.      The  corresponding  numerical  values  are 
read  from  the  matrix  equation    X  =  B     b  ,      where     b    is  the  vector  of 
constant  terms.      The  basis  inverse  is  used  also,    through  related  numbers 
called  simplex  multipliers,    to  test  the  current  solution  for  optimality, 
and  to  select  a  new  variable,    if  needed,    to  enter  the  solution. 

To  start  the  algorithm,    one  requires  a  feasible   solution  with    rn    non- 
zero coordinates  and  the  corresponding  basis  and  basis  inverse.      The 
optimal  solution  is  obtained  by  the  repeated  application  of  the  following 
four  steps,      (i)    A  test  of  whether  the  current  solution  is  optimal.      If 
not  -  (ii)     A  choice  of  the  new  variable  to  enter  the  solution,      (iii)    A 
choice  of  the  variable  which  is  to  leave  the  solution  (i.  e.  ,    to  become 
zero  m  the  next  stage),      (iv)     Inversion  of  the  new  basis   -  by  updating 
the  old  basis  inverse   -  and  then  back  to  (i)   . 


Row  Operations.  Interchanging  two  rows  of  a  matrix;  multiplying 
a  row  by  a  non-zero  constant;  adding  a  multiple  of  one  row  to  some 
other  row. 
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Shadow  Prices.        The  values  of  the  structural  (i.e.  ,    non-slack)  dual 

variables  at  optimality.      This  usage  is  appropriate  in  linear  programming  i 

problems  where  the  primary  variables     x_ ,  .  .  .  ,x       are  quantities  a 

In 

where  the  right-hand  side     b      of  the  typical  constraint      2   a    x    <    b 

i  ■       ij    J  =       i 

is  in  the   same  units  and  represents  the  quantity  available  system-wide 

of  some  particular  scarce  resource.     Since,    at  optimality,    the  objective 

function    a     may  be  expressed  as 


2     =     b  w    +  b    w     + 

1  i  L*         C* 


+  b     w 
m    m 


where  the     w.     are  the  structural  dual  variables,    we  see  that  the  profit 
i 

accruing  from  the  optimal  usage  of  the  resources     b  ,  .  .  .  ,  b         may  be 

broken  up  into    m    imputed  portions    b.w.     ;     and,    in  turn,      w      may  be 

11  \  ' 

regarded  as  an  imputed  unit  value  (shadow  price)  for     b.     .      Marginally, 
it  is  strictly  true  that 


w 


02 

~8tT 


i.  e.  ,    that  a  small  increase     Ab.     in  the  resource    b.   ,      keeping  the 

l  l 

other  resources  fixed,    will  result  in  an  increase  in  the  optimal    z     of 
amount    (Ab.)w.   .     It  follows  that  a  particular   shadow  price     w       is  zero 

if    2   a,  .x.     4     b,    at  optimality,    and  non-zero  if    2    an  .x.  =    b,      at  opti- 

J  j  J 

marity. 


Simplex.        A  convex  polyhedron  in  a  Eculidean  space  of  dimensionality 
equal  to  or  greater  than  the  number  of  linearly  independent  vertices. 
Simplexes  in  the  plane  are  points  and  line  segments;     simplexes  in  three - 
space  are  points,    line  segments,    and  triangles;  and  so  on.      The  dimen- 
sion of  a  simplex  is  the  dimension  of  the   smallest  sub-space  in  which  it 
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can  be  embedded;    e.g.,    a  triangle  in  a  space  of  dimension  three  or 
more  is  called  a  "simplex  of  dimension  two"  or,    more  briefly,    a  "two- 
simplex"  . 

Simplex  Method.      A  computational  routine  for  o  xt  tini  lj    the  o    ti  na.1 

solution  to  a  linear  programming  problem.      It  is  an  iterative  elimination 
procedure,    based  primarily  on  the  following  two  principles.      (1)     Elemen- 
tary row  operations  on  the  system  of  constraint  equations  leave  the   space 
of  feasible  solutions  unchanged.      (2)     The  number  of  non-zero  coordinate l-; 
in  an  optimal  solution  is  equal  to  (or  in  degenerate  cases  less  than)  the 
number  of  constraint  equations. 

At  each  stage  of  the  routine,    a  full  identity  matrix  must  be  present  as 
a  submatrix  of  the  coefficient  matrix  of  the  constraints.     If  not  present 
originally,    such  o.n  identity  matrix  must  be  introduced  artificially  before 
the  solution  process  can  begin.      The  columns  of  this  identity  matrix  serve 
to  mark  or  identify  the  current  non-zero  variables,    while  the  constant 
terms  on  the  right  specify  their  current  values.      The   simplex  algorithm 
provides  a  means  of  advancing  from  solution  to  solution  at  steadily  dimin- 
ishing cost,    until  the  optimum  is  reached,    while  maintaining  an  identity 
matrix  for  reading  off  the  interim  solution  at  each  stage. 

The  following  four  steps  comprise  one  iterative   stage,    or  cycle.      (1)  A 
test  of  whether  the  current  solution  is  optimal,      (ii)     If  not  optimal,    a 
choice  of  the  entering  variable   -  which  is  that  zero  variable  which  is  to 
become  non-zero  in  the  next  stage.      ( lii)     Choice  of  the  departing  variable 
which  is  to  become  zero  in  the  next  stage,      (iv)     Elementary  row  operations 
on  the  constraint  matrix,    so  as  to  update  the  position  of  the  identity  matrix. 
(This  step  is  called  the  "pivot  step".)     Then  back  to  (i)   . 
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Simplex  Multipliers.       The  components  of  a  vector  (  IT,,  IT,  ....  IT     ) 
±- s 1       c  m 

associated  with  the  current  vertex    X    at  a  stage  in  the  solution  of  a 

linear  programming  problem  with    m    constraints.     If  the  objective  func 

tion  is       2  c  x     ,     and  the  current  basis  is     B     ,      and  the  subscripts  of 
i    1 

the  basic  variables  are     i,,  i.,,  .  .  .  ,  i       ,     the   simplex  multipliers  are 

1     Z  m 

defined  by  the  equation 


(IT  ,  n    ,  .  .  .  ,  n     )  =  -  (c.   ,c.    ,  .  .  .  ,c.      )B 

12  m  Xl     xz 


-1 


The  vector  of  simplex  multipliers  has  an  interpretation  when  post- 
multiplied  by  each  column  of  the  constraint  matrix,    including  the  constant 

column.      Let    P      be  the  column  associated  with  the  coordinate    x    . 

J  J 

Then 


(n     •••II     )P 
1  m      k 


c         (x      in  basic  set) 

k       '    k 


and 


(II,   •  •  •  IT     )P,        =     A,     -  c,      (x       not  in  basic   set) 
1  m      k  k         k        k 


where     A       is  the  evaluation  associated  with  the  non-basic  variable    x 

k  k 

Also,    if    b    is  the  vector  of  constant  terms  and    z(X)     is  the  objective 
function  evaluated  at  the  vertex    X  ,      we  have 


(n    .  .-n    )b 

1  m 


a(X)     • 


Singular  Matrix.       A   square  matrix  whose  deterininant  is  zero,    and  thus 
whose  rank  is  less  than  its  dimension.     Singular  matrices  do  not  possess 
inverses. 


' 
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Slack.       The  difference  between  the  larger  and  smaller  members  of  an 
inequality.     If    u  <  v  ,     the  slack    s    is  defined  as     s  =  v-u  .     Slacks 
are  intrinsically  non-negative. 

Slack  Variable.         In  linear  programming,    an  auxiliary  variable  intro- 
duced to  convert  an  inequality  constraint  to  an  equation.      In  general,    there 
are  as  many  slack  variables  as  there  are  inequality  constraints;     each 
slack  appears  only  in  its  corresponding  equation;  and  its  coefficient  is 
one  of  the  numbers     +1    and     -1  . 

Slack  Vector,        The  column  vector  associated  with  a  slack  variable.      If 

the  slack  variable  occurs  in  the     r         constraint,    the  slack  vector  will 

th 
have     +1    or     -1    in  the     r         cell  and  zero  in  every  other  cell 

Solution.      In  linear  programming,    a  set  of  values  of  the  variables  which 
satisfy  all  the  given  constraints.      If  all  of  these  values  are  non-negative, 
the  solution  is  called  feasible;     if  one  or  more  are  negative,    it  is  called 
non-feasible  or  infeasible.      Solutions  are  otherwise  classified  as  basic 
(number  of  variables  with  non-zero  values  less  than  or  equal  to  the 
number  of  constraints)  and  non-basic  (number  of  non-zero  variables 
greater  than  the  number  of  constraints). 

Starting  Solutions.       Iterative  solutions  to  optimization  problems  often 
involve  progressing  from  feasible  solution  to  feasible  solution  at  steadily 
diminishing  cost  until  the  optimum  is  reached.      Such  algorithms  require 
that  a  starting  or  initial  solution  be  made  available  at  the  very  beginning, 


i 


i 


X-25 


as  an  input  to  the  computational  routine.      The  means  for  providing  this 
initiating  solution  is  generally  an  algorithm  in  its  own  right.      Examples: 
(1)    In  the  ordinary  simplex  method,    the  starting  solution  as  well  as  the 
initial  identity  basis  is  provided  by  the  introduction  of  slack  variables 
and  perhaps,    artificial  variables.      (2)    In  the  transportation  problem,    a 
starting  feasible  solution  can  be  obtained  by  means  of  the  so-called 
"northwest  corner  rule",    in  which  the   sources,    in  an  arbitrary  order, 
are  successively  emptied  into  the  destinations,    also  in  an  arbitrary  order, 
in  a  manner  dictated  by  the  requirements. 

Triangular  Matrix,        A   square  matrix  in  which  every  element  on  one 
side  of  the  diagonal  is  zero. 

Unit  Vector.        The  term  is  used  in  two  senses,    one  more  special  than 
the  other.      (1)    A  vector  of  unit  length.      (2)    A  vector  of  unit  length  with 
one  coordinate    +1    and  the  other  coordinates  all  zero. 

Vector  Space.       A   set  of  vectors  that  is  closed  under  the  operations  of 

vector  addition  and  multiplication  by  a  constant.     I.  e.  ,    given  any  subset 

of  vectors  in  the  vector  space,    their  sum  is  also  in  the  vector  space  and 

so  are  their  multiples  by  any  numerical  constant    k  .      The  set  of  all 

vectors    (x ,,  x_,  .  .  .  ,x   )    in  Euclidean  n-space  form  a  vector  space, 
1      2  n 

which  is  said  to  be  of  dimension    n     since  that  is  the  largest  number  of 
linearly  independent  vectors  that  can  be  found  in  this   space       Subsets 
of  vectors  in  this   space  may  also  form  vector  spaces,    but  of  smaller 
dimensionality;    e.g.,    the  set  of  all  k-multiples  of  a  fixed  vector  in  this 
space  is  a  vector  space  of  dimension  unity. 


< 
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Vertex.         A  point  of  a  convex  set  which  cannot  be  expressed  as  the 
convex  combination  of  other  points  in  the  set;    also  called  extreme  point. 
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